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Abstract  of 

A  Study  of  the  Processes,  Methods  and  Techniques  used  in 

Negotiated  or  "Best  Value"  Procurement  of  Construction  Services 

An  important  part  of  the  construction  process  is  awarding  the  contract  to  perform  the 
construction  in  a  timely  manner  to  a  qualified  contractor.  The  owner  must  choose  a  delivery 
method  and  a  selection  method.  The  four  basic  delivery  methods  are:  Traditional  Design-Bid- 
Build,  Construction  Management,  Design-Build,  and  Design-Build-Operate-Transfer  and  the  two 
most  commonly  used  selection  methods  are  Invitation  to  Bid  and  Request  for  Proposal  (Source 
Selection).  The  method  that  will  give  the  owner  the  most  flexibility  in  selecting  the  most 
qualified  contractor  is  the  Request  for  Proposal  or  Source  Selection  Process. 

There  are  several  basic  steps  in  the  Source  Selection  Process  that  ultimately  lead  to  the 
selection  of  the  "Best  Value"  Offeror.  In  the  private  industry  the  steps  are  left  to  the  discretion  of 
the  individual  owners.  For  Government  Agencies  there  are  applicable  rules  and  regulations  that 
apply  and  must  be  followed.  In  either  case,  the  basic  framework  of  the  process  is  the  same  for 
both  private  and  Government  "owners".  The  primary  difference  is  that  private  owners  can  enter 
into  contractual  relationships  in  any  matter  they  choose  as  long  as  it  is  not  fraudulent  and 
therefore  do  not  run  the  risk  of  the  process  being  stopped  by  a  protest.  On  the  other  hand,  the 
Government  runs  the  risk  of  the  process  being  stopped  by  a  protest  and  must  follow  all  the 
prescribed  rules,  regulations,  and  procedures  to  mitigate  this  risk. 

The  most  important  step  in  the  Source  Selection  Process  is  developing  Evaluation  Factors 
and  Subfactors  because  this  will  ultimately  determine  the  ease  and  cost  of  proposal  preparation 
for  the  offerors  and  the  ease  of  evaluating  proposals  for  the  evaluation  team.  The  second  most 
important  step  is  the  consistent  evaluation  of  the  proposals  received.  No  matter  how  complex  the 
evaluation  factors  are  if  the  evaluators  evaluate  the  proposals  consistently  and  adequately 
document  their  findings  the  probability  of  a  successful  protest  will  be  significantly  reduced. 

The  Source  Selection  Process  is  the  best  available  process  to  get  the  "best  value"  possible 
for  any  owner  organization.  It  helps  force  contractors  who  want  to  continue  to  win  contracts  to 
care  about  their  performance  ratings  on  projects,  helps  reduce  the  adversarial  relationships 
between  the  owner  and  the  contractor,  and  in  the  end  results  in  an  overall  higher  quality  project 
that  is  much  easier  to  administer. 
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PARTI 
Objective  and  General  Process 

1.0  Objective  of  the  Report 

The  objective  of  this  report  is  to  describe  and  analyze  the  process,  methods,  and  techniques 
owners  use  to  select  contractors  in  Negotiated  or  "Best  Value"  procurements  for  construction 
services. 

The  construction  process  from  concept  to  project  completion  has  many  steps.  Often  times 
once  the  decision  to  go  ahead  with  a  project  is  made  the  time  line  for  getting  the  project 
designed,  awarded,  and  constructed  is  critical  and  can  have  significant  economic  impact  if 
construction  completion  is  not  done  by  deadlines  and/or  within  the  most  reasonable  time 
possible.  For  example,  I  was  involved  in  two  school  projects  that  had  critical  construction 
completion  times  and  therefore,  it  was  crucial  they  were  awarded  on  time.  The  first  project  was 
the  design  build  construction  of  a  new  school  that  had  to  be  completed  before  the  lease  of  the 
existing  school  expired,  and  the  second  project  was  a  school  renovation  project  where  a 
significant  portion  of  the  work  had  to  be  done  during  the  summer  break.  Another  obvious 
example  is  for  every  day  of  delay  in  the  completion  of  a  hotel  or  apartment  complex  is 
significant  in  terms  of  lost  revenue. 

One  of  the  early  phases  of  the  construction  process  that  can  impact  the  time  cycle  and 
subsequent  success  (measured  in  quality  and  timeliness)  is  awarding  the  contract  in  a  timely 
manner  to  a  qualified  contractor.  This  report  will  focus  on  the  major  steps  in  the  selection  by 
briefly  describing  each  major  step,  comparing  different  agencies/organizations  methods  and 
procedure,  reviewing  case  studies,  and  interviewing  contractors  and  owner  personnel. 


Additionally,  the  report  will  focus  mostly  on  Government  selection  processes  because  it  is 
much  more  prone  to  protests  that  can  result  in  costly  legal  fees  and  considerably  delay  contract 
award  and  subsequent  project  completion.  As  per  Robert  Dorsey  in  CASE  STUDIES  IN 
BUILDING  DESIGN  &  CONSTRUCTION,  "private  owners  can  select  contractors  and  establish 
contractual  relationships  in  almost  any  manner,  short  of  fraud"  (Dorsey,  1999;  pg.  46).    Public 
owners  typically  fall  under  strict  regulations  and  procedures  and  when  a  selection  decision  does 
not  appear  fair,  dissatisfied  contractors  can  file  a  bid  protest  that  will  stop  the  entire  acquisition 
process  until  the  protest  is  resolved. 

2.0  General  Pre-Award  Process  and  Source  Selection 

2.1  Selecting  a  Method  to  Choose  a  Contractor 

Once  the  decision  is  made  to  pursue  the  project  the  owner  must  decide  on  the  delivery  system 
to  use.  There  are  four  basic  delivery  systems  with  variations  within  each  major  system: 
Traditional  Design-Bid-Build,  Construction  Management,  Design-Build,  and  Design-Build- 
Operate-Transfer.  Also,  at  this  point  the  owner  should  determine  the  selection  method  that  they 
intend  to  use.  There  are  two  basic  methods  commonly  used  in  the  United  States:  Invitation  to 
Bid  (IFB)  and  Request  for  Proposal  (RFP)  i.e.  Source  Selection.  The  invitation  to  bid  is 
typically  based  on  price  and  the  lowest  bidder  receives  the  award  provided  that  they  can  show 
that  they  have  adequate  financial  resources  to  get  the  project  done.  In  a  request  for  proposal  the 
owner  develops  guidelines  and/or  evaluation  factors  and  asks  the  contractor  to  submit  a  proposal 
for  completing  the  project  based  on  the  guidelines  in  the  owner's  request  for  proposal.  The 
owner  then  chooses  a  contractor  based  on  the  "best  value"  proposal  received. 

Until  recently  invitation  to  bid  has  been  the  most  popular  method  for  procuring  construction 
services.  However,  taking  the  lowest  bid  does  not  necessarily  result  in  the  lowest  final  cost  and 


it  does  not  typically  give  incentives  for  high  quality.  Therefore,  Requests  for  Proposals  have 
gained  much  wider  use  in  the  United  States  for  example,  the  Naval  Facilities  Engineering 
Command  (NAVFAC),  awarded  35%  of  its  contracts  using  RFP's  in  fiscal  year  1996,  40%  in 
FY  1997,  and  is  projected  to  reach  80%  by  the  end  of  the  FY  2000  (NFCTC,  1999). 

2.2  The  General  Source  Selection  Process 

Once  an  owner  chooses  to  use  Source  Selection  procedures  then  the  process  is  widely 
dependent  on  the  owner's  own  internal  policies  and  in  the  case  of  Federal  and  State  "Owners" 
the  laws  and  policies  that  apply  to  the  process.  Government  "Owners"  must  follow  stringent 
guidelines  and  almost  always  maintain  "Full  and  Open  Competition",  private  owners  have  great 
latitude  in  determining  how  they  will  select  contractors.  However,  the  major  steps  in  the  process 
are  similar  for  both  Government  and  Private  owners. 

The  basic  premise  is  that  the  owner  develops  a  request  for  proposal  (RFP)  and  then  sends  the 
RFP  out  to  contractors  asking  them  to  submit  proposals.  The  request  for  proposal  can  be  as 
simple  as  a  statement  of  work  and  some  general  guidelines  on  how  to  answer  the  proposal  to 
detailed  plans  and  specifications  with  very  detailed  instructions  on  how  to  answer  the  proposal. 
Often  times  there  will  be  a  request  for  two  submissions,  the  first  proposal  submitted  will  be  to 
pre-qualify  contractors  so  the  field  of  eligible  contractors  is  narrowed  down  for  the  final  and 
more  detailed  proposal  submissions.  The  detail  and  complexity  of  the  RFP  will  depend  on  the 
project  and  contract  delivery  method  i.e.  Design  Build,  Design-"Bid"-Build,  and/or  Construction 
Management,  and  the  rules  and  policies  of  the  owner  and/or  government  organization  that  is 
"buying"  the  project. 

The  owner  should  develop  the  RFP  with  specific  objectives  and/or  important  points  that  they 
want  the  contractor  to  address  that  will  help  the  owner  discriminate  between  proposals. 


Typically,  price  is  a  factor,  and  then  the  request  for  proposal  can  specify  other  non-price  related 
items  ranging  from  site  drawings  and  structural  plans  to  Past  Performance  and/or  a  Plan  to 
Manage  the  Project.  The  factors  should  relate  to  the  important  aspects  of  the  project  and/or  show 
the  contractor  is  capable,  and  how  capable,  he/she  is  to  perform  the  project. 

The  number  of  contractors  the  RFP  is  sent  to  depend  on  the  owner.  In  the  case  of  some 
organizations,  such  as  the  Walt  Disney  World  Company,  the  requests  are  sent  out  to  a  selected 
group  of  contractors  who  have  already  pre-qualified  with  Disney  through  a  pre-qualification 
process.  For  specific  projects  Disney  sends  out  a  request  for  proposal  to  5  or  6  contractors  on  the 
pre-qualified  list  based  on  contractor  skill  and  Disney's  perception  of  the  workload  of  the 
contractor  (Greenwood,  Lee).  In  the  case  of  the  Federal  Government  the  request  has  to  be  sent 
to  the  Commerce  Business  Daily  to  be  publicly  published  and  the  public  is  invited  to  submit  a 
proposal.  The  Florida  Department  of  Transportation  also  uses  an  initial  "pre-qualification 
process"  but  the  process  is  limited  to  the  contractors  assets  and  does  not  address  other  factors 
such  as  Past  Performance,  personnel  qualifications,  etc.    In  fact  they  pre-qualify  about  400 
contractors  per  year  and  when  a  contract  is  ready  for  "bid"  letters  of  interest  are  sent  to  the  list  of 
contractors  on  the  pre-qualified  list  (Moore,  Juanita). 

Once  Proposals  are  received,  the  owner  and/or  owners  representatives  meet,  review  the 
proposals,  and  ideally  select  the  contractor  that  will  provide  the  "best  value"  performance. 


3.0  Definitions 

Competitive  Range.  All  proposals  that  the  contracting  officer  determines  to  be  the  most 

highly  rated  based  on  the  cost  and  noncost  evaluation  factors  stated  in  the  solicitation. 

Evaluation  Factors  and  Subfactors.  Descriptions  of  those  aspects  of  a  proposal  that 

will  be  evaluated  quantitatively  or  qualitatively  to  assess  which  proposal  can  best  meet 

the  Government's  requirements  as  described  in  the  solicitation. 

Evaluation  Standards  (also  called  evaluation  criteria).  A  baseline  level  of  merit  or 

acceptability  used  for  measuring  how  well  or  whether  an  offeror's  response  meets  the 

solicitation's  requirements  with  respect  to  an  evaluation  factor  or  subfactor. 

Rating.  The  application  of  a  scale  of  words,  colors,  or  numbers,  used  in  conjunction  with 

narrative,  to  denote  the  degree  to  which  the  proposal  has  met  the  standard  for  a  non-cost 

factor. 

Source  Selection/Evaluation  Plan.  The  document  that  explains  how  proposals  are  to  be 

solicited  and  evaluated  to  make  the  selection  decision.  It  is  the  Government's  plan  for 

how  it  intends  to  acquire  its  needs. 

Source  Selection  Process.  The  process  of  soliciting  and  evaluating  proposals  for  award 

in  a  competitively  negotiated  environment. 

Trade  Offs.    The  process  of  choosing  something  that  is  a  balance  between  price  and 

technical  merit. 


Part  II 

Analysis  and  Description  of  the  Major 

Steps  in  a  Source  Selection 

4.0  Introduction 

The  Source  Selection  Process  consists  of  several  basic  steps  that  ultimately  result  in  an  award 
to  the  "Best  Value"  Offeror.  In  the  private  industry  the  steps  are  left  to  the  discretion  of  the 
individual  owners.  For  Government  Procurements  there  are  applicable  rules  and  regulations  that 
apply  and  must  be  followed  throughout  the  process.  In  either  case,  the  basic  framework  of  the 
process  is  the  same  for  both  private  and  Government  organizations. 

The  following  will  primarily  focus  on  Government  source  selections  because  the  rules  and 
regulations  must  be  followed.  If  the  process  is  not  followed  the  end  result  could  be  a  protest  that 
could  result  in  a  complete  work  stoppage  and  much  lost  time.  As  stated  earlier  Private  industry 
does  not  run  this  risk. 

5.0  The  Source  Selection  Team 

Once  a  decision  is  made  to  use  the  Source  Selection  process  a  team  must  be  established  to 
help  develop  the  Request  for  Proposal  (RFP),  evaluate  the  proposals,  and  recommend  and  select 
the  best  value  contractor.  It  is  very  important  to  form  the  team  early  in  the  process.  The  Source 
Selection  Team  members  should  have  input  into  the  selection  factors.  The  level  of  involvement 
of  the  Team  members  and  the  complexity  of  the  team  organization  is  usually  determined  based 
on  the  aspects  of  the  project  itself  (AMC  715-3,  1998;  NFCTC,  1998). 

The  Team  can  be  very  structured  and  have  many  groups  and  subgroups  to  simply  one  or  two 
people  evaluating  the  proposals.  AMC  Pamphlet  715-3,  CONTRACTING  FOR  BEST  VALUE, 
A  BEST  PRACTICES  GUIDE  TO  SOURCE  SELECTION,  has  an  example  of  different  source 


selection  teams.  Figure  1  is  an  example  of  a  complex  Source  Selection  Team.  Figure  2  is  an 
example  of  a  simple  evaluation  team. 


Source  Selection 
Complex  Source  Selection  Team 


Contracting  Officer 
Business  Advisor  to 
the  SSA 


Source  Selection  Authority  (SSA) 
FINAL  DECISION 


Source  Selection  Advisory  Council 
Optional  for  very  complex  source  selections 


Source  Selection  Evaluation  Team 

Evaluates  proposals  against  RFP  Factors  and  Subfactors 

Assists  SSA  in  comparative  analysis,  if  required 


Advisors 
Government 
(if  needed) 


Technical  Subgroup 
Evaluates  Technical  Merit 
and  Proposal  Risk 


Advisors 

Non  Government 

(if  needed) 


Cost  Subgroup 
Evaluates  Cost 


Past  Performance  Subgroup 
Evaluates  Performance  Risk 


Figure  1 
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Source  Selection 
Non-Complex  Source  Selection  Team 


Figure  2 


The  Naval  Facilities  Engineering  Command  has  a  typical  Source  Selection  organization  as 
shown  in  figure  3  (NFCTC,  1998). 


Source  Selection  Authority 
Makes  Final  Decision 


Source  Selection  Board 
Makes  a  recommendation  to  the 
Source  Selection  Authority 

. 

Technical  Evaluation  Board 
Evaluates  all  technical  and 
performance  factors 

Price/Cost  Evaluation  Board 
Evaluates  price  factors 

Figure  3 
All  of  the  above  organizations  are  Department  of  Defense  Organizations  however  committees 
and  selection  groups  are  formed  for  other  organizations  as  well. 

n 


All  of  the  organizations  I  talked  to  included  the  "end-user"  in  the  selection  process. 
According  to  J.T.  McAferty  from  the  University  of  Florida's  Planning  Department  they  short  list 
their  consultants,  (A&E  and  Construction  Managers),  and  after  the  short  list  the  specific 
Department  the  facility  is  being  constructed  for  within  the  University  has  a  very  significant  role 
in  making  the  final  selection.  Also,  Lee  Greenwood  stated  that  they  consult  with  the 
construction  management  branch  and  the  "end-user"  to  choose  the  contractor.  It  is  my 
experience  in  the  Navy  that  many  of  the  "end-users"  are  not  involved  in  the  detailed  review  of 
proposals  but  rather  they  receive  briefs  from  the  chairman  of  the  Technical  and/or  Source 
Selection  Board.  However,  this  very  much  depends  on  the  customer,  type  of  project,  and 
importance  of  the  project. 

It  is  important  to  maintain  the  integrity  of  the  team  throughout  the  process.  Once 
evaluations  are  started  major  changes  to  the  team  can  be  detrimental  to  the  selection.  Obvious 
changes  can  and  should  occur  if  a  conflict  of  interest  is  discovered  or  other  unforeseen 
circumstances  arise  among  individual  members.  For  example,  on  one  selection  I  was  involved 
with,  after  proposals  were  received  and  the  technical  evaluation  board  met  and  started  reviewing 
proposals  one  of  the  technical  board  members  noticed  that  his  wife's  aunt  held  an  executive 
position  on  one  of  the  offeror's  board  of  directors.  The  technical  board  member  notified  the 
contracting  officer  and  was  appropriately  removed  from  the  technical  evaluation  board.  In  this 
case  there  were  still  several  remaining  members  on  the  technical  evaluation  board  and  the  board 
successfully  completed  the  evaluations. 

However,  care  should  be  taken  if  major  changes  in  the  technical  evaluation  boards 
membership  or  procedures  are  changed  as  shown  in  the  following  case  study. 
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5.1  Case  Study  United  International  Investigative  Services  vs.  United  States 

In  United  International  Investigative  Services  vs.  United  States,  41  Fed.  CI.  312  (1998)  the 

Court  ruled  in  favor  of  the  plaintiff  when  only  2  of  the  6  members  were  called  back  to  evaluate 

the  proposals.  The  Contracting  Officer  felt  that  the  Technical  evaluations  were  flawed  and 

directed  the  Technical  Evaluation  Board  Chairman  to  revisit  the  proposals.  The  TEB  Chairman 

directed  two  of  the  six  TEB  members  to  review  the  proposals  and  validate  the  scores.  However, 

the  record  is  not  clear  whether  one  or  two  of  the  TEB  members  actually  did  this.  Whatever,  the 

case  the  TEB  member(s)  who  revisited  the  proposal  made  significant  changes  in  the  scoring. 

The  court  ruled  in  favor  of  the  plaintiff  because  the  court  felt  in  part  that 

The  group  discussion,  as  demonstrated  above,  was  necessary  in  order  to  ventilate  the 
evaluators'  opinions  regarding  each  proposal.  During  these  discussions,  evaluators  could 
have  explained  their  individual  reasons  for  ascribing  a  score  to  a  particular  proposal. 
Moreover,  the  TEB,  as  a  group,  would  then  have  been  able  to  arrive  at  a  consensus 
regarding  that  particular  aspect  of  an  offeror's  proposal. 

Although  minor  irregularities  or  errors  in  the  procurement  process  are  not  sufficient 
grounds  to  warrant  judicial  intrusion  to  overturn  a  procurement  decision,  Grumman  Data 
Sys.  Corp.  v.  Dalton,  88  F.3d  990,  1000  (Fed.  Cir.  1996),  the  violation  in  this  case  was  not 
minor.  Rather,  the  violation  described  above  deprived  plaintiff  of  the  opportunity  to  have 
its  proposal  considered  fairly  and  honestly.  Thus,  the  decision  to  have  two  evaluators 
circumvent  the  consensus  and  discussion  requirements  of  the  Technical  Evaluation  merits 
judicial  intrusion. 

6.0  Source  Selection  Approach 

Once  the  decision  is  made  to  do  a  Source  Selection  the  type  of  approach  to  use  must  be 
determined.  There  are  several  types  of  approaches  and  some  examples  of  different  approaches 
are  described  below: 

6.1  Pre-qualification  -  Submit  Proposals 

In  this  approach  the  owner  asks  contractors  to  submit  a  pre-qualification  proposal.  The  pre- 
qualification  proposal  usually  only  asks  for  general  qualifications,  past  performance  on  similar 
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projects,  and  other  company  specific  factors.  Then  the  owner  pre-qualifies  usually  three  to  five 
of  the  original  offerors.  The  contractors  who  are  pre-qualified  will  then  get  sent  the  full  request 
for  proposal  including  project  specific  evaluation  criteria. 

This  approach  is  usually  more  applicable  to  complex  acquisitions,  design/build  type  projects, 
and/or  simpler  acquisitions  when  there  is  an  expectation  that  an  excessive  number  of  proposals 
will  be  received. 

The  advantages  to  this  are: 

a.  It  reduces  the  number  of  "complex"  or  detailed  proposals  that  the  owner  has  to 
evaluate  and  makes  selection  easier. 

b.  It  reduces  the  number  of  contractors  who  are  submitting  expensive  proposals  and 
eliminates  contractors  who  would  not  have  a  very  good  chance  at  getting  an 
award  before  they  waste  too  many  resources  developing  a  proposal. 

The  disadvantages  are: 

a.  It  adds  another  step  in  the  process  and  perhaps  more  time  in  the  selection  process. 

b.  For  simple  projects  it  may  not  be  appropriate. 
6.2  Submit  Proposals 

The  owner  develops  specific  criteria  and  asks  contractors  to  submit  proposals. 

The  advantages  are: 

a.  It  eliminates  a  step  and  can  be  done  in  one  evaluation  phase  and  therefore  in  some 
cases  can  be  done  faster  than  the  pre-qualification  method. 

The  disadvantages  are: 

a.  It  can  result  in  getting  an  excessive  number  of  proposals  and  make  it  very 
difficult  for  evaluating  the  best  value  proposal. 

b.  It  can  result  in  too  many  contractors  investing  resources  in  developing  a  proposal 
that  they  have  no  chance  to  a  very  slim  chance  of  winning. 
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6.3  Maintain  List  of  Pre-qualified  Companies  ask  for  proposals 

Some  private  companies  maintain  a  list  of  pre-qualified  contractors.  For  example,  the  Walt 

Disney  World  Corporation  maintains  a  database  of  contractors  who  have  been  pre-qualified  to  do 

work  for  them.  The  contractors  must  request  to  get  on  the  list  and  then  once  they  are  approved  to 

go  on  the  list  Disney  will  consider  them  for  projects  of  the  appropriate  size  and/or  trade  that  the 

contractor  is  qualified  to  perform.  For  specific  projects  Disney  invites  5  to  6  companies  to 

submit  proposals  on  the  project  from  their  pre-qualified  list  of  contractors  (Greenwood,  Lee). 

For  Federal  projects  a  list  of  pre-qualified  contractors  such  as  Disney  maintains  would  not  be  in 

compliance  with  10  U.S.C.  2304  and  41  U.S.C.  253,  which  requires  Federal  Agencies  to 

Maintain  Full  and  Open  Competition.  Therefore,  Federal  Agencies  would  not  be  allowed  to  do 

this  unless  under  very  specific  circumstances. 

The  advantages  to  this  are: 

a.    Build  a  working  relationship  with  contractors  who  know  that  if  they  perform  well 
they  will  have  a  good  chance  for  follow  on  work. 

The  disadvantage  is: 

a.  May  eliminate  very  qualified  contractors  who  are  not  pre-qualified. 

6.4  Lowest  Price  Technically  Acceptable 

This  method  is  most  likely  a  Government  unique  process.  It  is  defined  in  the  Federal 
Acquisition  Regulation  FAR  Part  15. 101-2.  It  is  a  process  where  contractors  are  asked  to  submit 
price  and  a  technical  proposal  and  the  evaluator's  rate  the  technical  proposal  as  either  pass  or 
fail.  It  is  used  when  the  best  value  is  expected  from  the  lowest  price,  technically  acceptable 
proposal. 

The  advantages  is: 

a.  It  is  much  easier  to  evaluate  proposals  and  therefore  can  result  in  a  faster  award. 
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The  disadvantages  is: 

a.  It  does  not  allow  for  trade  offs.  A  much  more  qualified  contractor  may  have 
submitted  a  proposal  for  just  slightly  more  money  than  a  much  less  qualified 
contractor  and  not  get  the  award 

For  example,  I  was  involved  in  a  $500,000  warehouse/cold  storage 
renovation  project  using  this  method  and  a  contractor  with  an  excellent 
proposal  submitted  a  price  that  was  less  than  1%  higher  than  a  contractor 
who  submitted  a  proposal  that  just  met  requirements.  The  contract  award 
had  to  go  to  the  "just  met  requirements"  contractor  even  though  it  was  clear 
the  Government  would  have  received  better  value  with  the  slightly  higher 
cost. 

6.5  Design  Build  Adjusted  Score 

This  method  was  described  by  Ken  Leuderalbert,  Florida  Department  of  Transportation, 
Manager  of  Quality  Initiatives.  The  method  is  used  solely  for  selecting  contractors  for  Design- 
Build  projects  in  the  Florida  Department  of  Transportation.  Basically,  it  is  similar  to  the  Pre- 
qualification  -  Submit  Proposals,  method  described  above  but  the  technical  evaluation  panel 
assigns  a  percentage  score  to  the  technical  proposals  from  Phase  II.  Then  the  submitted  price  is 
divided  by  the  technical  score  and  the  lowest  price  wins.  The  table  is  a  sample  (not  from  a  real 
project)  of  the  "Adjusted  Score". 


Bid  Price 

Technical 
Score 

Calculation 

Adjusted  Bid 

Offeror  1 

1,000,000 

80% 

1,000,000/.  8 

1,250,000 

Offeror  2 

1,200,000 

95% 

l,200,000/.95 

1,263,157 

Offeror  3 

900,000 

65% 

900,000/.65 

1,384,615 

Offeror  1  would  be  awarded  this  project. 
The  advantages  are: 

a.  It  eliminates  another  level  of  proposal  review  where  price  and  technical  factors 
are  considered  together  to  determine  best  overall  value. 

b.  It  helps  alleviate  appearances  of  favoritism    (The  technical  evaluation  panel  does 
not  see  the  price  during  technical  proposal  reviews). 
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The  disadvatages  are: 

a.    It  does  not  allow  for  trade  offs  between  cost  and  technical  merit  and  therefore 
the  "Best  Overall  Value"  contractor  may  not  get  the  award. 

7.0  The  Source  Selection  Plan 

Once  the  decision  is  made  to  use  the  Source  Selection  method  a  plan  should  be  prepared  to 
define  the  process  and  evaluation  methods  to  be  used.  This  plan  actually  starts  with  the 
formation  of  the  team  and  choosing  the  delivery  method.    For  small  non-complex  acquisitions 
the  plan  can  be  very  simple  but  for  larger  more  complex  acquisitions  it  can  be  very  complex. 

In  Federal  Acquisitions  a  source  selection  strategy  or  acquisition  plan  must  be  developed. 
Federal  Acquisition  Regulation  Part  15.303(b)  states  in  part  that  one  of  the  requirements  of  the 
source  selection  authority  is  to,  "Approve  the  source  selection  strategy  or  acquisition  plan,  if 
applicable,  before  solicitation  release."  Many  agencies  have  developed  guidelines  and 
requirements  for  what  is  commonly  called  the  "Source  Selection  Plan". 

The  major  purposes  of  most  Source  Selection  Plans  include; 

•  Defining  a  specific  approach  for  soliciting  and  evaluating  proposals. 

•  Describing  the  evaluation  factors  and  subfactors,  their  relative  importance,  and  the 
methodology  used  to  evaluate  proposals. 

•  Providing  essential  guidance  to  the  solicitation  developers,  especially  for  putting 
together  the  solicitation  sections  dealing  with  proposal  preparation  and  evaluation. 

•  Serving  as  a  charter  and  guide  for  the  source  selection  team  on  the  roles  of  the 
members  and  the  conduct  of  the  entire  source  selection  from  proposal  evaluation, 
through  the  cost/price/technical  tradeoff,  award  decision,  and  debriefing  (AMC 
Pamphlet  715-3,  1998). 

Appendix  A  is  a  blueprint  for  a  Source  Selection  plan  from  AMC  Pamphlet  715-3  and  shows 
recommendations  for  the  contents  and  basic  layout  of  a  source  selection  plan 
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Basically,  as  stated  earlier  the  Source  Selection  Plan  becomes  a  map  for  the  entire 
procurement  process  from  team  selection  to  debriefing  of  unsuccessful  offerors  and  should  be 
the  document  used  to  develop  the  solicitation  and  evaluate  proposals. 

7.0  Determining  What  and  How  to  Evaluate  Proposals 

8.1  Evaluation  Factors  and  Subfactors 

Evaluation  Factors  and  Subfactors  are  descriptions  of  those  aspects  of  a  proposal  that  will  be 
evaluated  quantitatively  or  qualitatively  to  assess  which  proposal  can  best  meet  the  requirements 
as  described  in  the  solicitation.  They  are  also  commonly  referred  to  as  evaluation  criteria.  They 
are  the  basis  of  what  the  proposals  will  be  evaluated  on  and  how  the  selection  will  be  made 

For  Federal  Acquisitions  all  of  the  evaluation  factors  and  subfactors  must  clearly  be  stated  in 
the  solicitation  and  the  Source  Selection  Plan   The  Federal  Acquisition  Regulation  (FAR  Part 
15.304)  requires  Cost  and  Past  Performance  to  be  evaluation  factors  (unless  under  very  specific 
circumstances),  and  depending  on  dollar  value  the  extent  to  which  Small  and/or  Small- 
Disadvantage  Businesses  are  used.  The  Federal  Acquisition  Regulation  Part  15.304  clearly 
states  the  requirements  for  evaluation  factors  and  subfactors  and  is  therefore  included  in  part 
below.  Key  elements  of  this  section  are  colored. 

15.304  Evaluation  factors  and  significant  subfactors. 

(a)  The  award  decision  is  based  on  evaluation  factors  and  significant  subfactors  that  are 
tailored  to  the  acquisition. 

(b)  Evaluation  factors  and  significant  subfactors  must— 

(1)  Represent  the  key  areas  of  importance  and  emphasis  to  be  considered  in  the  source 
selection  decision;  and 

(2)  Support  meaningful  comparison  and  discrimination  between  and  among  competing 
proposals. 

(c)  The  evaluation  factors  and  significant  subfactors  that  apply  to  an  acquisition  and  their 
relative  importance,  are  within  the  broad  discretion  of  agency  acquisition  officials,  subject 
to  the  following  requirements: 
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(1)  Price  or  cost  to  the  Government  shall  be  evaluated  in  every  source  selection  (10  U.S.C. 
2305(a)(3)(A)(ii)  and  41  U.S.C.  253a(c)(l)(B))  (also  see  Part  36  for  architect-engineer 
contracts); 

(2)  The  quality  of  the  product  or  service  shall  be  addressed  in  every  source  selection 
through  consideration  of  one  or  more  non-cost  evaluation  factors  such  as  past  performance, 
compliance  with  solicitation  requirements,  technical  excellence,  management  capability, 
personnel  qualifications,  and  prior  experience  (10  U.S.C.  2305(a)(3)(A)(i)  and  41  U.S.C. 
253a(c)(l)(A));and 

(3)(i)  Except  as  set  forth  in  paragraph  (c)(3)(iv)  of  this  section,  past  performance  shall  be 
evaluated  in  all  source  selections  for  negotiated  competitive  acquisitions  expected  to 
exceed  $1,000,000. 

(ii)  Except  as  set  forth  in  paragraph  (c)(3)(iv)  of  this  section,  past  performance  shall  be 
evaluated  in  all  source  selections  for  negotiated  competitive  acquisitions  issued  on  or  after 
January  1,  1999,  for  acquisitions  expected  to  exceed  $100,000.  Agencies  should  develop 
phase-in  schedules  that  meet  or  exceed  this  schedule 

(iii)  For  solicitations  involving  bundling  that  offer  a  significant  opportunity  for 
subcontracting,  the  contracting  officer  must  include  a  factor  to  evaluate  past  performance 
indicating  the  extent  to  which  the  offeror  attained  applicable  goals  for  small  business 
participation  under  contracts  that  required  subcontracting  plans  (15  U.S.C. 
637(d)(4)(G)(ii)). 

(iv)  Past  performance  need  not  be  evaluated  if  the  contracting  officer  documents  the  reason- 
past  performance  is  not  an  appropriate  evaluation  factor  for  the  acquisition. 

(4)  The  extent  of  participation  of  small  disadvantaged  business  concerns  in  performance  of 
the  contract  shall  be  evaluated  in  unrestricted  acquisitions  expected  to  exceed  $500,000 
($1,000,000  for  construction)  subject  to  certain  limitations  (see  19.201  and  19.1202). 

(5)  For  solicitations  involving  bundling  that  offer  a  significant  opportunity  for 
subcontracting,  the  contracting  officer  must  include  proposed  small  business 
subcontracting  participation  in  the  subcontracting  plan  as  an  evaluation  factor  (15  U.S.C. 
637(d)(4)(G)(i)). 

Individual  agencies  also  may  require  different  evaluation  factors  to  be  considered  and  limit 
the  number  of  factors  that  can  be  used.  For  instance,  for  certain  acquisitions  Sub-Contracting 
Plans  and  Sub-Contracting  with  Small  and/or  Small  Disadvantaged  Businesses  is  a  required 
element  in  more  stringent  detail  than  prescribed  in  the  Federal  Acquisition  Regulation. 
Interesting  to  note,  The  Disney  World  Company  also  considers  Small  and  Small  Disadvantaged 
Businesses  when  selecting  contractors  (Greenwood,  Lee). 

The  Naval  Facilities  Engineering  Command's  Policy  is  to  minimize  the  number  of  non 
cost/price  evaluation  factors  with  normally  three  to  five.  NAVFAC  recommends  that  typical 
evaluation  criteria  include: 
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1 .  Past  Performances 

2.  Subcontracting  with  Small  and  Small  Disadvataged  Business 

3.  Technical/Management  Approach 

4.  Key  Personnel  (NFCTC,  1998) 

These  major  factors  seem  to  be  very  common  with  all  of  the  agencies  I  interviewed  including, 
Florida  Department  of  Transportation.,  University  of  Florida,  and  the  Army  Corps  of  Engineers. 
AMC  Pamplet  715-3  summarizes  the  purpose  and  importance  of  Technical  evaluation  factors 
Technical  evaluation  factors  address  the  proposal's  technical  and  performance 
efficiency  and  it  is  important  to  get  the  correct  factors.  The  basic  requirements  for 
noncost  evaluation  factors  are: 

•  A  reasonable  expectation  of  variance  among  proposals  in  that  area. 

•  A  variance  that  can  be  measured  quantitatively  or  qualitatively. 

•  The  factor  must  be  a  true  discriminator. 

An  evaluation  factor  should  be  chosen  only  if  your  requirements  warrant  a 
comparative  evaluation  of  that  area    The  simplest  way  to  assess  a  potential  evaluation 
factor  is  to  ask:  "Will  superiority  in  this  factor  provide  value  to  the  Government  and  is 
the  Government  willing  to  pay  more  for  that  superiority?" 

Selecting  the  right  evaluation  factors  is  one  of  the  most  important  decisions  you  will 
make  in  designing  your  evaluation  process.  We  are  often  faced  with  the  triple  problem 
of  less  time,  less  funds,  and  fewer  available  personnel  to  devote  to  source  selections.  If 
you  don't  concentrate  on  what's  important  in  selecting  the  best  value  offeror  you  could 
end  up  with  the  evaluation  team  wasting  a  lot  of  time  and  effort  looking  at  issues  that 
don't  differentiate  between  offerors.  This  can  also  result  in  a  weak  evaluation  that 
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doesn't  give  the  source  selection  authority  the  information  needed  to  make  a  good 
selection. 
It  is  my  experience  that  selecting  the  right  evaluation  factors  is  not  just  one  of  the  most 
important  decisions  but  the  most  important  decision  in  designing  the  evaluation  process. 
Evaluation  Factors  must  not  only  meet  the  criteria  stated  above  but  they  also  must  be  written  in  a 
clear  and  concise  manner  so  the  contractor  clearly  understands  what  the  Government  wants.  I 
have  been  in  many  circumstances  where  unclear  evaluation  factors  have  led  to  difficulties  and 
undo  hardship  on  the  evaluation  committee  and  offerors. 
8.1.1  Selecting  Evaluation  Factors  Do's  and  Don'ts 

The  importance  of  selecting  appropriate  evaluation  factors  can  not  be  stressed  enough 
According  to  Vernon  J.  Edwards,  a  respected  consultant  in  Government  contracting:  "I  would 
suggest  that  the  single  most  effective  thing  acquisition  managers  can  to  do  streamline  the  best 
value  source  selection  process  is  to  improve  their  choices  of  evaluation  factors  for  award." 
(Savare,  Matthew,  www.monomouth.army.mil/cecom/ac/hb/sourceselection  1 2.html) 

Almost  all  of  the  organizations  that  I  interviewed  said  that  when  they  started  doing  Source 
Selections  they  asked  for  too  much  detail    However,  it  seems  from  my  interviews  that  most 
everyone,  including  the  contractors  were  satisfied  with  the  evaluation  factors  and/or  criteria  that 
is  currently  being  used  and  did  not  see  a  need  for  major  improvements.  The  following  is  a  list 
of  general  lessons  learned  and  or  advice  from  the  different  organizations  and/or  my  personal 
experience. 
•     Only  ask  for  the  information  that  will  discriminate  between  proposals  and  can  be  evaluated 

This  has  been  stated  before  however,  it  is  worth  repeating.  Specifically,  make  sure  that  the 
evaluation  factors  will  actually  be  of  value  in  discriminating  between  proposals  and/or  actually 
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be  an  indicator  of  performance.  Paul  Nishaizaki,  an  architect  from  the  NAVFAC  Pacific 
Division,  used  the  following  example  as  an  element  under  Past  Performance  that  was  not 
particularly  valuable.  He  said  that  he  has  seen  solicitations  that  ask  for  the  data  on  completed 
contracts  that  included  among  other  things:  Original  award  amount,  final  contract  amount, 
original  contract  completion  date,  and  final  completion  date  and  that  this  information  was  not 
particularly  useful.  For  most  projects  the  original  amount  and  completion  date  is  different  from 
the  final  contract  amount  and  completion  date  and  as  an  evaluator  there  was  no  way  of  knowing 
what  the  causes  were  for  the  differences  without  having  personal  knowledge  and/or  requiring  the 
contractor  to  submit  some  kind  of  narrative  description. 
•     Insure  Criteria  is  not  inconsistent  or  spread  out  too  much  in  the  proposal 

Mr.  Richard  Palmer,  from  "C"  Construction  Company,  stated  that  sometimes  it  is  difficult  to 
really  nail  down  all  of  the  criteria  because  it  is  spread  out  all  through  the  request  for  proposal. 
Mr.  Larry  Fisk,  Naval  Facilities  Engineering  Commands  Southern  Division,  also  stated  that  this 
caused  difficulties  in  some  of  their  proposals.  In  order  to  help  alleviate  his  organization  now 
develops  a  paragraph  that  identifies  special  project  requirements  and  all  of  the  evaluation  factors 
refer  back  to  that  paragraph.  This  process  is  described  in  more  detail  in  section  8.4. 

I  was  involved  in  a  Design-Build  Golf  Course  project  that  had  criteria  spread,  throughout  the 
proposal.  There  were  some  minimum  requirements  that  included  in  part:  pump  size  and  number, 
reservoir  size  and  dimensions,  number  of  sprinkler  heads  and  types,  number  of  holes  that  could 
be  taken  out  of  operation  at  a  time,  and  the  control  system  to  be  used.  In  addition,  there  were 
desirables  that  if  included  would  be  considered  favorably.  An  example  of  one  of  the  desirables 
was  watering  within  some  number  of  yards  to  the  tee  but  closer  was  better. 
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The  minimum  requirements  were  spelled  out  in  the  solicitation  but  they  were  "spread  out" 
throughout  the  solicitation.  The  technical  evaluation  board  and  it  seemed  offerors  had  some 
problem  keeping  all  of  the  criteria  organized  and  too  make  sure  they  met  all  of  the  minimum 
requirements  while  trying  to  meet  the  desirables.  The  technical  evaluation  board  had  to  spend  a 
lot  of  time  going  through  the  RFP  and  back  to  the  proposals  to  make  sure  that  all  of  the 
minimum  requirements  were  met.  The  Technical  Evaluation  Board  all  agreed  at  the  conclusion 
of  the  technical  board  that  it  would  have  been  much  better  to  have  a  succinct  list  of  project 
requirements  on  one  or  two  pages. 
•     Do  not  be  afraid  to  limit  the  amount  of  information 

It  is  my  opinion  that  this  is  most  applicable  to  specific  factors  like;  Past  Performance,  Safety, 
Quality  Control,  and  other  factors  that  are  not  as  prone  to  innovations  or  creative  solutions  that 
would  require  significant  explaining.  I  was  involved  in  a  RFP  that  was  for  the  repair  of  a 
Typhoon  damaged  hangar.  Most  of  the  work  was  structural  with  some  interior  repairs.  One  of 
the  sub  factors  under  Past  Performance  was  to  demonstrate  that  the  offeror  had  experience  on 
similar  type  projects.  Several  of  the  offerors  submitted  extensive  lists  of  projects  and  I 
remember  specifically  that  one  of  the  offerors  submitted  a  list  of  80  projects  ranging  from 
runway  lighting  repairs  to  concrete  wharf  construction  and  repairs.  The  technical  evaluation 
board  had  to  wade  through  this  information  and  determine  what  was  applicable  and  what  was 
not.  After  that  solicitation  our  office  started  limiting  the  number  of  project  to  show  experience  to 
no  more  than  10  or  15  projects   Naval  Facilities  Engineering  Commands  Southern  Division, 
limits  the  number  of  projects  to  3  to  5  that  relate  specifically  to  the  identification  of  special 
project  requirements  paragraph  and  also  state  in  their  proposals  that  they  desire  not  to  exceed  a 
certain  number  of  pages  (Fisk,  Larry). 
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In  my  opinion  another  example  of  a  place  to  limit  pages  is  the  submission  of  corporate  Safety 
Plans  or  Quality  Management  Plans.  These  documents  are  usually  large  and  can  constitute  a 
significant  portion  of  the  bulk  of  a  proposal  and  do  not  necessarily  provide  clear  discrimination 
between  offerors.  Most  of  the  time  corporate  Safety  Plans  and  Quality  Control  Plans  contain 
basically  the  same  information  from  one  company  to  the  next,  however  some  companies  do  have 
unique  aspects  within  these  plans.  But,  for  the  most  part  they  are  the  same.  It  is  difficult  for  a 
technical  evaluator  to  wade  through  all  of  the  material  in  these  plans  and  pick  out  unique  aspects 
and/or  valuable  aspects  that  set  contractors  apart  from  one  another.  Our  office  changed  our 
policy  of  asking  for  complete  plans  limiting  the  information  to  unique  aspects  and  project 
specific  aspects  of  Quality  and  Safety  Plans  and  limited  the  number  of  pages  that  would  be 
accepted.  It  may  not  always  be  appropriate  to  limit  information,  like  most  factors  it  depends  on 
the  project  type  and  complexity. 
•     Make  sure  evaluation  factors  and  submittal  requirements  are  clear  and  specific. 

Mr.  Nishaizaki,  (NAVFAC  Southdiv),  stated  that  he  has  sometimes  seen  that  after  proposals 
are  received  that  the  government  wasn't  clear  on  the  types  of  submittals  and/or  clear  enough  on 
the  evaluation  factors  by  the  content  of  the  different  proposals   Mr  Garicia  of  J.  A.  Jones 
Construction,  stated  that  his  office  had  submitted  proposals  in  the  past  where  they  clearly 
misinterpreted  the  evaluation  factors  and  that  he  would  like  to  see  presentations  and/or 
interviews  to  insure  a  mutual  understanding  of  requirements. 

It  appears  to  me  that  most  of  the  organizations  that  I  talked  to  had  already  gone  through  the 
learning  curve  for  developing  and  selecting  evaluation  factors  and  were  doing  much  better  at 
putting  clear  evaluation  factors  and  submittal  requirements  in  their  solicitations. 
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8,2  Weighting  the  Factors  and  Subfactors 

Weighting  of  evaluation  factors  and  subfactors  is  the  "ranking"  of  importance  of  one  factor 
over  other  factors.  The  Federal  Acquisition  Regulation  requires  that  the  relative  importance  of 
factors  and  subfactors  be  clearly  stated  (FAR  PART  15.304(d)).  This  is  not  required  in  private 
industry.  The  Walt  Disney  World  Company  does  not  typically  rank  their  evaluation  factors;  they 
review  their  proposals  in  the  entirety  and  choose  the  best  contractor  with  no  formal  ranking  of 
evaluation  factors  (Greenwood,  Lee).  For  Federal  procurements  the  relative  importance  of 
evaluation  factors  and  subfactors  is  usually  established  by  priority  statements,  numerical 
weighting,  or  a  combination  of  these.  The  FAR  also  clearly  requires  that  the  difference  between 
cost/price  and  other  non  cost/price  be  clearly  stated  at  a  minimum  as  follows: 

(d)  All  factors  and  significant  subfactors  that  will  affect  contract  award  and  their  relative 
importance  shall  be  stated  clearly  in  the  solicitation  (10  U.S.C.  2305(a)(2)(A)(i)  and  41 
U.S.C.  253a(b)(l)(A))  (see  15.204-5(c)).  The  rating  method  need  not  be  disclosed  in  the 
solicitation.  The  general  approach  for  evaluating  past  performance  information  shall  be 
described. 

(e)  The  solicitation  shall  also  state,  at  a  minimum,  whether  all  evaluation  factors  other  than 
cost  or  price,  when  combined,  are— 

(1)  Significantly  more  important  than  cost  or  price; 

(2)  Approximately  equal  to  cost  or  price;  or 

(3)  Significantly  less  important  than  cost  or  price  (10  U.S.C.  2305(a)(3)(A)(iii)  and  41 
U.S.C.  253a(c)(l)(C)). 

Naval  Facilities  Contracts  Training  Center  lists  examples  of  Priority  Statements.  Two  of  the 

seven  examples  listed  are  included  below: 

Example  1 .  Priority  Statement 

Proposals  will  be  evaluated  on  technical/business  qualifications  and  price.  The 
technical/business  qualifications  factors  will  rank  as  the  highest  factors  in  this 
procurement,  and  are  significantly  more  important  than  price,  although  price  will  also  be 
important  in  the  evaluation  process.  Within  technical  qualifications,  there  are  three 
subfactors.  These  are  corporate  capabilities,  key  personnel  and  past  performance  data. 
Past  Performance  is  of  greater  importance  than  the  other  two  subfactors.  Key  personnel 
and  corporate  capability  are  of  equal  importance. 


25 


Example  2.  Numerical  Relationship 

The  Government  will  make  award  to  the  responsible  offeror(s)  whose  offer  conforms  to 
the  solicitation  and  is  most  advantageous  to  the  Government,  cost  or  price  and  technical 
factors  listed  below  considered. 
The  evaluation  factors  listed  below  are  in  descending  order  of  importance: 

(1)  Past  performance  on  similar  projects 

(2)  Technical  and  Management  Approach 

(3)  Experience  on  similar  projects 

(4)  Qualifications  of  key  personnel 

(5)  Price/cost  (NFCTC,  1998) 

Whatever  the  priority  statement  is  it  must  show  the  relationship  of  the  importance  of  the 
individual  factors.  This  becomes  important  when  selecting  the  "best  value"  contractor.  If  the 
priority  statement  is  weak  or  ambiguous  it  can  cause  different  interpretations  among  the  Source 
Selection  Team  and  contractors  and  some  confusion  in  choosing  the  "best  value"  contractor 

8.3  Standards  for  Evaluation  Factors 

Evaluators  must  be  able  to  determine  the  relative  merit  of  each  proposal  with  respect  to  the 
evaluation  factors.  Evaluation  standards  provide  guides  to  help  evaluators  measure  how  well  a 
proposal  addresses  each  factor  and  subfactor  identified  in  the  solicitation.  Standards  permit  the 
evaluation  of  proposals  against  a  uniform  objective  baseline.  This  helps  to  minimize  bias  and 
promotes  consistency  in  the  evaluation  by  ensuring  that  the  evaluators  evaluate  each  proposal 
against  the  same  baseline  (AMC  Pamphlet  715-3,  1998). 

Two  examples  of  evaluation  standards  follow: 

This  standard  shall  be  met  when  the  offeror  provides  evidence  of  a  documented  and 
functioning  in-house  training  program  for  the  handling,  transport  and  disposal  of 
hazardous  waste  in  accordance  with  EPA  and  State  guidelines  and  regulations.  (NFCTC, 
1998) 

STANDARD  FOR  PAST  PERFORMANCE  has  been  met  when  the  offeror  provides  at  least 
10  of  any  combination  of  the  following:  Performance  evaluations  for  past  contracts,  letters 
of  recognition,  letters  of  appreciation,  recognition  awards,  and  any  other  client 
documentation  that  recognizes  acceptable  performance  on  past  projects. 
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When  our  office  first  started  doing  Source  Selections  on  a  regular  basis  we  did  not  include 
evaluation  standards.  I  was  involved  with  about  6  of  these  Source  Selections  and  after  we 
incorporated  evaluation  standards  it  did  in  fact  eliminate  some  of  the  confusion  and  personal 
interpretation  problems  that  we  were  having  in  earlier  solicitations. 

8,4  Using  Identification  of  Special  Project  Requirements  Paragraphs 

The  Southern  Engineering  Field  Division,  Naval  Facilities  Engineering  Command 

(SOUTHDIV)  use  identification  of  special  project  requirements  as  one  of  the  basis  to  evaluate 

offeror's  proposals.  The  solicitation  has  a  paragraph  that  clearly  states  the  special  project 

requirements  and  then  the  evaluation  factors  are  developed  so  they  relate  back  to  the  special 

requirements  (Fisk,  Larry). 

For  example,  they  issued  a  solicitation  for  a  2  Phase  Design  Build  Wharf  Utilities  Project  in 

Mayport  Florida.  The  special  project  requirements  paragraph  said  that  experience  with  electrical 

distribution  systems  among  other  things  was  required.  The  evaluation  factors  were  all  related 

back  to  the  special  project  requirements  paragraph;  past  performance,  technical  qualifications, 

management  approach,  and  small  business  subcontracting  efforts  (Fisk,  Larry).  One  of  the 

offerors  filed  a  GAO  Protest,  Dvrocher  Dock  &  Dredge  Black  &  Veatch,  A  Joint  Venture, 

B280853,  November  24,  1998,  in  short  stating  that  they  were  improperly  evaluated  and  should 

have  been  ranked  higher  and  included  in  the  second  phase.  However,  the  GAO  denied  their 

protest  stating  in  part: 

The  TEB  assigned  Durocher's  proposal  a  rating  of  marginal  under  this 
factor.  Under  the  two  subfactors  evaluated  in  this  area-design  team 
and  construction  team-offerors  were  instructed  to  submit  key 
personnel  staffing  that  demonstrated  experience  in  the  elements 
included  in  the  Specialized  Project  Requirements  provision  of  the  RFP. 
Offerors  were  also  required  to  provide  qualifications  and  staffing 
capability  to  support  the  project. 
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Under  the  design  team  subfactor,  the  TEB  found  that  only  one  individual  Durocher 
proposed  as  "key  personnel"  had  participated  in  any  of  the  projects  Durocher  described 
in  its  proposal  (and  even  that  individual  had  participated  in  only  one  project  described 
in  the  proposal),  and  identified  this  as  a  weakness  in  the  proposal.  The  TEB  further 
found  that  the  proposed  key  personnel  demonstrated  no  waterfront  electrical 
distribution,  steam,  or  fuel  system  experience  and  considered  this  a  weakness  in  the 
proposal.  Under  the  construction  team  subfactor,  the  TEB  found  that  the  proposed  key 
personnel  demonstrated  no  waterfront  electrical  distribution  experience,  and  that  the 
waterfront  experiences  listed  for  key  structural  personnel  were  minimal  in  scope.  The 
protester  does  not  rebut  any  of  the  TEB's  findings  in  this  regard. 

In  its  proposal,  Durocher  identified  the  firm  of  [DELETED]  as  a  mechanical  team- 
member  and  [DELETED]  as  the  electrical  installation  team  member.  The  TEB  noted, 
however,  that  the  protester  did  not  include  any  personnel  data  for  [DELETED],  and 
found  that  the  proposal  demonstrated  no  specific  diesel  fuel  or  steam  work  experience 
by  construction  project  managers,  superintendents,  or  quality  assurance  personnel 

This  GAO  case  summary  is  included  in  its  entirety  in  Appendix  B.  It  should  also  be  noted  the 
case  is  also  an  excellent  example  of  a  TEB  supporting  their  decision  with  a  solid  narrative. 

The  below  is  a  partial  example  of  a  special  projects  requirement  provision  of  a  solicitation 

currently  in  Phase  II  of  the  solicitation. 

This  project  involves  the  site-adapting  design  and  construction  of  a  standard  LIS  Navy  T- 
10  aircraft  jet  engine  test  cell  facility  to  support  intermediate  aircraft  engine  maintenance 
for  F/A  -18  squadrons  at  Marine  Corps  Air  Station,  Beaufort,  SC.  These  test  cell  facilities 
more  closely  resemble  a  working  piece  of  machinery  than  a  typical  brick  and  mortar  type 
project.  They  are  designed  and  constructed  to  operate  without  failure  when  subjected,  to  the 
stresses  induced  by  continuous  operation  of  high-powered  aircraft  jet  engines  within  the 
engine  run  room.  The  test  cell  construction  involves  fabrication  and  installation  of  complex 
facility  equipment  and  instrumentation.  The  facility  equipment  and  instrumentation  to  be 
provided  closely  integrates  with  NAVAIR  furnished  and  installed  equipment, 
instrumentation  test  systems  and  a  variety  of  aircraft  jet  engines.  The  construction  of  a  T- 
10  jet  engine  test  cell  involves  modular  prefabricated  steel  and  sheet  metal  type  fabrication. 
The  major  components  of  the  facility  are  jet  engine  run  room,  primary  and  secondary  air 
inlets,  movable  primary  air  inlet  turning  vanes,  exhaust  augmenter  and  exhaust  stack,  all 
constructed  with  acoustically  treated  panels  and  materials.  The  augmenter  will  be  the  US 
Navy  T-10  standard  test  cell  air  cooled  round  type  used  in  other  similar  facilities  around 
the  world.  The  test  facility  and  especially  the  augmenter  are  subjected  to  continuous 
vibration,  acoustical  loading  and  pressure  variations.  In  addition  the  augmenter  is  subjected 
to  a  wide  range  of  variations  in  internal  temperatures  from  ambient  up  to  2,800+F  and 
airflow  velocities  in  excess  of  2,500  feet  per  second.  The  quality  of  the  fabrication  of  the 
facility  and  especially  the  augmenter  system  is  extremely  critical  to  ultimate  success  of  this 
test  facility  project.  The  manufacture  and  installation  of  the  prefabricated  aircraft  engine 

28 


test  facility  is  desired  to  be  performed  by  a  manufacturer  with  an  adequate  history  of 
proven  successful  experience  in  the  fabrication  of  complex  close  tolerance  prefabricated 
steel/metal  type  structures  who  is  regularly  engaged  in  this  type  of  work,  has  the 
equipment,  tools,  technical  personnel  and  experienced  shop  personnel  for  the  fabrication  of 
close  tolerance  prefabricated  structures,  and  demonstrates  adequate  shop  space  for 
fabrication  and  shop  assembly/factory  demonstration  of  the  test  facility  indoors.  Other 
complex  systems  include  air  start  system,  fuel  and  oil  preservation  systems,  engine  test 
trailer  restraint  system,  proof  load  test  system,  fire  protection,  industrial  waste,  power  and 
lighting.  Supporting  facilities  will  involve  replacement  of  the  existing  air  start  compressor, 
installation  of  20,000-gallon  underground  fuel  tank,  fuel  oil  separator,  civil  site  work  and 
utilities.  The  estimated  price  is  between  $5,000,000  and  $10,000,000.  Proposals  will  be 
evaluated  using  Two  Phase  Design-Build  Request  for  Proposal  procedures  that  will  in 
Phase  II  result  in  an  award  of  a  firm-fixed  price  design-build  contract  to  the  responsible 
offeror  whose  proposal,  conforming  to  the  solicitation,  is  the  "Best  Value"  to  the 
Government,  price  and  technical  factors  considered.  Phase  I  is  the  pre-qualification  phase 
wherein  offerors  will  be  evaluated  on:  FACTOR  A  -  Past  Performance  for  Design  and 
Construction;  FACTOR  B  -  Small  Business  Subcontracting  Effort  (applies  to  small  and 
large  business);  and  FACTOR  C  -  Technical  Qualifications  in  Design  and  Construction 
and  Factor  D  -  Management  Approach.  The  most  qualified  offerors  (not  to  exceed  5)  will 
be  selected  to  proceed  into  Phase  II .  The  Phase  II  Request  for  Proposals  will  be  issued 
only  to  the  pre-qualified  offerors.  Phase  II  will  include  the  Price  Proposal  and  an  additional 
technical  factor  Technical  Solutions,  which  will  require  submission  of  a  limited  design 
solution  for  the  project  site  work  and  other  sub-factors  that  define  the  quality  of 
construction,  (www.esol.navfac.navy.mil,  N62467-99-R-1055) 

According  to  Mr.  Larry  Fisk,  this  paragraph  was  designed,  to  help  streamline  the  Source 
Selection  Process  and  so  far  has  worked  well.  It  appears  that  this  approach  helps  offerors  insure 
that  their  proposals  are  focused  on  important  factors  and  helps  evaluators  focus  on  what  is 
important  in  an  offeror's  proposal. 

8.5  Establishing  a  Rating  Method 

The  rating  method  is  the  "score"  that  each  evaluation  factor  will  receive.  Each  rating  must 
have  a  definition.  There  are  3  common  rating  methods  when  developing  a  proposal  and  AMC 
Pamphlet  715-3  is  a  summary  of  the  methods  and  the  descriptors: 
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Typical  Ratings  and  Descriptors 


Each  rating  must  have  a  definition. 


TECHNICAL  MERIT  ratings  reflect  the 
its  proposal,  to  perform  the  requirements 
color)  to  evaluate  technical  merit 


government's  confidence  in  each  offeror's  ability,  as  demonstrated  in 
stated  in  the  RFP.  Choose  one  method  (e.g.,  numerical,  adjectival,  or 


NUMERICAL     ADJECTIVAL 


90-100 


Excellent 


COLOR 


Blue 


DEFINITION 


Proposal  demonstrates  excellent  understanding  of 
requirements  and  approach  that  significantly  exceeds 
performance  or  capability  standards.  Has  exceptional 
strengths  that  will  significantly  benefit  the 
Government. 


80-89 


Good 


Green 


Proposal  demonstrates  good  understanding  of 
requirements  and  approach  that  exceeds  performance 
or  capability  standards.  Has  one  or  more  strengths  that 
will  benefit  the  Government. 


70-79 


Satisfactory 


Yellow 


Proposal  demonstrates  acceptable  understanding  of 
requirements  and  approach  that  meets  performance  or 
capability  standards.  Acceptable  solution.  Few  or  no 
strengths.      


60-69 


Marginal 


j  Amber 


Proposal  demonstrates  shallow  understanding  of 
requirements  and  approach  that  only  marginally  meets 
performance  or  capability  standards  necessary  for 
minimal  but  acceptable  contract  performance. 


<60 


Unsatisfactory 


Red 


Fails  to  meet  performance  or  capability  standards. 
Requirements  can  only  be  met  with  major  changes  to 
the  proposal 


Rating  systems  which  use  adjectives  or  colors  are  usually  the  most  successful  because  they 
allow  maximum  flexibility  in  making  the  tradeoffs  among  the  evaluation  factors  (AMC  Pamphlet 
715-3).  However,  many  organizations  use  a  numerical  system.  Ms.  Claudia  Hunley,  Army 
Corps  of  Engineers  Jacksonville,  stated  that  the  numerical  system  is  easier  to  explain  and  easier 
for  members  of  Technical  Evaluation  Boards  to  understand  and  for  this  reason  they  use  a 
numerical  rating  system. 

A  narrative  definition  must  accompany  each  rating  in  the  system  so  that  evaluators  have  a 
common  understanding  of  how  to  apply  the  rating.  For  example,  a  rating  of  excellent  (or  blue  or 
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90-100)  could  be  defined  as  meaning  an  outstanding  approach  to  specified  performance  with  a 
high  probability  of  satisfying  the  requirement.  What  is  key  in  using  a  rating  system  in  proposal 
evaluations,  is  not  the  method  or  combination  of  methods  used,  but  rather  the  consistency  with 
which  the  selected  method  is  applied  to  all  competing  proposals  and  the  adequacy  of  the 
narrative  used  to  support  the  rating  (AMC  Pamphlet  715-3,  1998). 
Each  major  rating  is  described  below: 

Adjectival 

Adjectives  (such  as  excellent,  good,  satisfactory,  marginal,  and  unsatisfactory)  are  used  to 

indicate  the  degree  to  which  the  offeror's  proposal  has  met  the  standard  for  each  factor 
evaluated.  Adjectival  systems  may  be  employed  independently  or  in  connection  with  other  rating 
systems.  Holmes  &  Edwards 

Color  Coding 

This  system  uses  colors  to  indicate  the  degree  to  which  the  offeror's  proposal  has  met  the 

standard  for  each  factor  evaluated.  For  instance,  the  colors  blue,  green,  yellow,  amber,  and  red 
may  indicate  excellent,  good,  satisfactory,  marginal,  or  unsatisfactory  degrees  of  merit, 
respectively. 

Numerical 

This  system  assigns  point  scores  (such  as  0-10  or  0-100)  to  rate  proposals.  This  rating  system 

generally  allows  for  more  rating  levels  and  thus  may  appear  to  give  more  precise  distinctions  of 
merit.  However,  numerical  systems  can  have  drawbacks  as  their  apparent  precision  may  obscure 
the  strengths,  weaknesses,  and  risks  that  support  the  numbers.  Therefore,  some  organizations  do 
not  permit  the  use  of  numerical  rating  systems. 

Narrative 

Narrative  is  used  in  conjunction  with  a  rating  system  to  indicate  a  proposal's  strengths, 

weaknesses,  and  risks.  Adjectival,  color,  and  numerical  ratings  must  be  supported  with  narrative 
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statements.  Narrative  statements  can  describe  the  proposals'  relative  strengths,  weaknesses,  and 
risks  to  the  source  selection  authority  in  a  way  that  adjectives,  colors,  and  numbers  alone  cannot. 
A  narrative  is  required  when  evaluation  standards  are  being  applied,  when  a  comparison  of 
proposals  is  being  made,  and  when  a  cost/technical  tradeoff  is  conducted.  The  narrative  provides 
a  reasonable  and  rational  basis  for  the  selection  decision  (AMC  Pamphlet  715-3). 

Different  organizations  may  or  may  not  require  the  use  of  specific  rating  methods  and  the 
three  rating  methods  above  are  not  the  only  rating  methods  that  can  be  used.  Examples  of 
different  requirements  are:  NAVFAC  Policy  is  to  use  an  adjectival  system,  the  Air  Force  uses  a 
color  system,  and  the  Florida  DOT  uses  a  numeric  system  that  ends  in  an  overall  percentage 
score  for  their  technical  proposals 

8.6  Summary 

Once  a  solicitation  is  issued  the  most  important  thing  is  to  follow  what  is  said  in  the 
solicitation  in  terms  of  evaluation  factors,  evaluation  factor  weight,  standards,  and  the  rating 
method  and  support  the  selection  decision  with  adequate  documentation.  Developing  the  factors 
will  have  the  most  significant  impact  in  terms  of  ease  of  evaluation,  time  required  to  evaluate 
proposals,  and  reduction  in  the  probability  of  a  protest.  Making  it  easier  to  evaluate  and  identify 
strengths  and  weaknesses  by  choosing  the  right  evaluation  factors,  weights,  standards,  and  rating 
methods  will  ultimately  make  it  easier,  faster,  and  safer  (from  protests)  to  make  an  award  to  the 
Best  Value  contractor. 

9.0  The  Solicitation 

The  solicitation  provides  all  of  the  information  the  offeror  needs  to  understand  what  you  are 
buying,  how  you  are  buying  it,  and  how  you  will  select  who  to  buy  it  from.  This  information 
includes:  the  work  requirements;  the  terms  and  conditions;  evaluation  factors  and  significant 
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subfactors;  the  relative  importance  of  the  factors  and  subfactors;  instructions  to  offerors, 
including  whether  award  might  be  made  without  discussions;  and  other  exhibits  and  attachments 
When  read  as  a  whole,  the  solicitation  should  convey  to  the  offerors  a  clear  understanding  of 
what  you  are  buying  and  the  areas  where  technical  and  cost  tradeoffs  can  be  made  in  their 
proposals  to  best  satisfy  the  Government  requirements. 

The  solicitation  is  what  the  contractors  will  see  and  what  they  will  use  to  develop  their 
proposals.  When  developing  the  solicitation  consistency  across  the  board  is  important   There 
should  not  be  conflicts  between  the  description  of  what  we  are  buying,  instructions  on  how  to 
prepare  a  proposal,  and  guidance  on  important  factors/subfactors  ,  and  the  ground  rules  for  the 
evaluation  (AMC  Pamphlet  715-3,  1998). 

10.0  Evaluating  Proposals 

Evaluating  proposals  is  the  process  in  which  each  proposal  is  evaluated  in  detail  against  the 
evaluation  factors  and  subfactors  and  the  requirements  set  forth  in  the  solicitation,  and  assigning 
a  rating  with  a  supportive  narrative.  It  is  not  an  analysis  of  proposals  against  each  other.  For 
Government  projects  it  is  very  important  that  evaluations  be  fair,  thorough,  and  impartial.  It  is 
also  important  to  stress  detailed  analysis.  The  Technical  Evaluation  Board  must  be  detailed  in 
their  review  and  reach  a  consensus  in  their  ratings  of  the  proposals 

Additionally,  as  stated  in  section  8.5,  the  technical  evaluation  board  must  provide  narrative 
statements  to  establish  a  written  record.  Numerical  scores  and  other  rating  techniques  are  not 
conclusive  data  to  make  the  source  selection  decision.  Only  evaluations  and  ratings 
substantiated  by  documented  specific  strengths,  weaknesses,  and  risks  can  be  credible  and 
justifiable   General  terms  such  as  "weak",  "poor",  or  "excellent"  must  be  supported  with 
specific  reasons  as  to  why  the  proposal  is  "weak",  "poor",  or  "excellent"  in  relation  to  the 
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standard  for  the  specific  factor  and/or  subfactor  being  evaluated.  A  technical  evaluation  that 
does  not  substantiate  proposal  strengths,  weaknesses,  and  risks  will  not  be  able  to  stand  up  in  a 
protest  (AMC  715-3,  1998). 

10,1  GAP  Case  Studies 

I  have  chosen  two,  of  many  GAO  cases,  that  demonstrate  the  importance  of  the  evaluating  the 
proposals  according  to  what  is  prescribed  in  the  solicitation.  It  is  worth  noting  that  there  are  a 
large  number  of  cases  that  demonstrate  this  and  that  this  is  probably  the  single  biggest  factor  for 
protests  that  are  sustained  against  the  government.  For  more  extensive  review  of  GAO  cases  the 
GAO's  web  site  contains  a  complete  list  of  all  the  cases  they  have  reviewed  since  1996.  The  site 
is  www.gao.gov.  Each  of  the  case  summaries  described  below  are  included  in  Appendix  B. 

10,1,1  Unfair  Evaluation  of  Proposals 
GAO  Case  Tidewater  Homes  Realty,  Inc.,  B-274689,  Dec.  26,  1996 

In  this  case  the  GAO  found  that  there  was  an  unequal  evaluation  of  the  proposals  in 
solicitation  by  the  Department  of  Housing  and  Urban  Development  (HUD)  and  sustained  the 
protest.  They  found  that  HUD's  technical  evaluation  panel  rated  one  of  the  technical  evaluation 
factors  the  protestors  received  a  score  of  6  while  the  awardee's  proposal  received  a  score  of  24 
points   The  technical  evaluation  panel  asserted  that  the  protestor's  proposal  lacked  detail  and 
clarity  in  the  factor,  however,  when  the  GAO  reviewed  the  proposals  they  found  that  the 
information  in  each  proposal  was  essentially  the  same.  For  another  technical  factor  the  agency 
downgraded  the  protestors  proposal  because  they  said  it  was  vague  in  certain  areas.  However 
GAO's  review  of  the  factor  showed  just  the  opposite. 

It  is  difficult  to  determine  what  led  to  the  protest  and  missed  interpretation  of  the 
proposals.  It  could  have  been  a  lack  of  detailed  review,  not  following  the  evaluation  factors, 
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and/or  bias  toward  one  contractor  on  the  part  of  the  Technical  Evaluation  Panel.  However,  it 
clearly  shows  the  need  to  do  a  detailed  analysis  of  the  proposals  so  key  factors  or  points  are  not 
missed. 

10,1.2  Not  Following  Established  Criteria,  inadequate  documentation 
GAO  Case  Lloyd H.  Kessler,  Inc.,  B-284693,  May  24,  2000 

In  this  case  the  agency  did  not  perform  the  evaluation  as  stated.  In  fact  the  agency's  actual 
evaluation  differed  from  that  stated  in  the  RFP  in  that  the  relative  weights  for  experience  and 
past  performance  were  reversed  and  experience  was  assigned  70  points  and  past  performance 
was  assigned  30  points.  Additionally,  the  agency  rated  very  highly  for  experience  in 
bioengineered  slope  protection  experience  when  in  the  RFP  it  was  never  mentioned  to  be 
important   Additionally,  in  the  notes  of  this  case  (  #5)  the  GAO  alludes  to  the  importance  of 
documenting  the  selection  decision  and  the  GAO  states,  "even  beyond  the  agency's  failure  to 
disclose  the  actual  evaluation  plan  in  the  RFP,  the  documentation  supporting  the  evaluation  is 
minimal,  such  that  it  would  be  difficult  to  determine  the  reasonableness  of  the  evaluation,  even  if 
the  RFP  were  not  defective." 

10,2  Oral  Presentations 

Oral  presentations  are  another  method  of  presenting  proposals.  It  must  be  determined  early  in 
the  process  if  oral  presentations  are  going  to  be  used  and  clear  and  precise  instructions  must  be 
put  into  the  solicitation  for  conducting  the  oral  presentations.  Also,  the  evaluation 
criteria/factors  should  be  explicitly  stated  and  clear  so  offerors  know  what  information  is 
expected.  (AMC  Pamphlet  715-3,  1998;NFCTC,  1998) 

For  Federal  Contracting  it  is  very  important  to  make  sure  the  content  of  the  oral  presentations 
are  documented.  There  are  many  sustained  cases  where  during  oral  presentations  the 
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government  did  not  adequately  document  the  presentations  and  therefore  there  is  not  an  adequate 

record  of  performance.  A  recent  example  is  J&J Maintenance,  Inc.,  B-284708.2;  B-284708.3, 

June  5,  2000.  The  contract  was  for  housing  maintenance  and  service  and  the  GAO  sustained  the 

protest  stating  in  part, 

In  this  procurement,  the  proposals  addressing  the  quality  factor  consisted  of  oral 
presentations,  briefing  slides,  and  resumes.  No  written  technical  proposals  were  allowed, 
jf]  While  the  RFP  stated  that  the  agency  could  videotape  and/or  record  the  oral 
presentations,  the  agency  did  not  do  so.  RFP  amend.  9,  at  L-6.  In  this  case,  the  offerors' 
slides  and  the  evaluators'  notes  are  the  only  record  of  what  was  included  in  the  oral 
presentations.  We  recognize  the  SSO's  discretion  to  decide  the  method  and  level  of  detail 
of  the  record  of  an  oral  presentation  and  that  the  use  of  offerors'  slides  and  government 
notes  are  two  of  several  possible  methods  of  documenting  an  oral  presentation  under  FAR 
§  15.102(e).  Here,  however,  the  slides  and  notes  do  not  present  sufficient  information  to 
determine  if  the  evaluations  of  J&J's  and  D&Z's  proposals  were  conducted  in  a  reasonable 
and  equal  manner.  The  slides  provide  only  an  outline-in  most  cases,  the  slides  include 
general  headings  only—and  do  not  describe  what  was  included  in  the  approximately  2-hour 
oral  presentations  and  question-and-answer  sessions.  J^]  The  evaluators'  notes  are  not 
summaries  of  the  presentation  but,  for  the  most  part,  are  selective  comments  which  are 
sketchy  and  provide  almost  no  elaboration,  or  even  a  description,  of  what  was  contained  in 
the  portion  of  the  oral  presentation  being  commented  upon  by  the  evaluator. 

In  addition,  the  consensus  evaluation  ratings  consist  only  of  numerical  scores  for  each 
offer  in  each  evaluation  element,  without  any  accompanying  narratives  to  show  how  the 
panel  reconciled  the  divergent  opinions  of  the  individual  evaluators  on  any  particular  facet 
of  an  oral  presentation  or  how  the  panel  achieved  a  consensus  rating  for  any  element  of  the 
evaluation  There  is  no  evidence  in  the  consensus  ratings  sheets  (or  elsewhere  in  the 
record)  that  the  evaluators  discussed  the  strengths  and  weaknesses  of  the  proposals  or  used 
any  other  method  to  reconcile  their  differences  to  arrive  at  the  consensus  scores  Agency 
Report,  Tabs  I,  J,  Consensus  Ratings  for  Quality  Proposals  (D&Z  and  J&J,  respectively). 

We  understand  that  oral  presentations  are  an  effective  way  to  streamline  the  source 
selection  process  and  to  enhance  the  agency's  understanding  of  an  offeror's  approach,  and 
we  do  not  believe  that  the  FAR  requirement  for  documentation  described  above  necessarily 
limits  this  flexibility.  FAR  §§  15.102(e)  and  15.308  do  not  require  a  particular  method  of 
establishing  a  record  of  what  was  said  by  offerors  during  oral  presentations  or  place  an 
excessive  burden  on  contracting  agencies  in  recording  oral  presentations.  Whatever 
method  is  chosen,  FAR  §§  15.102(e),  15-305(a)  and  15-308  do  establish  an  obligation  to 
provide  a  reasonably  adequate  record  of  such  presentations  and  the  evaluation  thereof 
Such  a  record  permits  our  Office  to  perform  a  meaningful  review  of  the  agency's  selection 
decision.  Here,  the  oral  presentations  constituted  the  offerors'  entire  technical  proposals 
(the  only  written  portions  of  proposals  related  to  past  performance)  The  record  of  the  oral 
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presentations  and  the  evaluation  is  so  sketchy,  that  we  have  no  means  to  determine,  based 
upon  the  record  before  us,  the  reasonableness  of  the  agency's  selection. 


The  other  key  elements  in  conducting  oral  presentations  are: 

•  Limit  the  time  for  each  presentation. 

•  Some  of  the  proposal  should  be  in  writing,  ie  hard  data  such  as  performance  history  or 
anything  that  will  be  included  in  the  contract  documents. 

•  If  exchanges  are  going  to  be  allowed  during  the  presentation  then  it  is  important  that 
they  are  fair  and  meaningful  for  each  offeror  (AMC  Pamphlet  715-3,  1998). 

The  use  of  oral  presentations  depends  on  the  agencies/organizations  that  I  talked  to.  For 

instance,  Ms  Claudia  Hunley,  ACOE,  said  they  use  them  when  they  feel  that  it  is  appropriate, 

and  said  as  a  contracting  officer  she  liked  using  oral  presentations   Mr  Phenix  Palmer  of  FDOT 

and  Mr.  Larry  Fisk  (SLIOLITHDIV)  said  they  prefer  not  to  use  oral  presentations.  The  LIniversity 

of  Florida  extensively  uses  oral  presentations  when  using  the  Source  Selection  Process. 

10,3  Summary 

The  technical  evaluation  of  the  offeror  proposals  is  the  foundation  of  the  source  selection 

process  and  the  pending  contract  award.  It  is  also  the  source  of  the  biggest  areas  for  contractors 

to  protest.  The  development  of  good  evaluation  factors  and  subfactors  will  significantly  effect 

the  complexity  of  the  evaluation  process  and  ease  of  evaluation   The  key  points  in  evaluating 

proposals  are: 

•  Evaluations  must  be  fair,  thorough,  and  impartial. 

•  Document  specific  strengths,  weaknesses,  and  risks  of  the  proposed  approach. 

•  It  must  be  done  in  accordance  with  the  requirements  stated  in  the  RFP  and  the  Source 
Selection  Plan. 

•  The  evaluation  must  provide  enough  detail  for  determining  merits  between  proposals 
and  enough  detail  for  the  contracting  officer  to  make  a  best  value  determination  and  or 
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establish  a  competitive  range  of  offerors  for  further  consideration.  (AMC  Pamphlet  715- 
3,  1998;  NFCTC,  1998) 

Even  if  the  evaluation  factors  are  not  very  good  and/or  hard  to  evaluate  if  the  above  points  are 

followed  the  risk  of  a  successful  protest  are  low. 

1 1.0  Exchanges  with  Offerors  After  Receipt  of  Proposals 

In  Federal  Contracting  there  are  basically  three  types  of  exchanges  as  per  FAR  15.306; 
clarifications  and  award  without  discussions  (Clarifications),  communications  with  offerors 
before  establishing  the  competitive  range  (Communications),  and  exchanges  with  offerors  after 
establishment  of  the  competitive  range  (Discussions). 

Clarifications  are  informal  and  limited  to  correction  of  minor  deficiencies,  clerical  errors,  as 
well  as  to  give  the  offeror  the  opportunity  to  resolve  adverse  past  performance  issues. 
Communications  are  exchanges  with  an  offeror  whose  past  performance  is  the  only  factor 
preventing  them  from  getting  into  the  competitive  range.     Discussions  are  where  the  offerors 
will  be  allowed  to  revise  their  proposals.  It  is  the  actual  "negotiating"  proposals  and  prices  can 
be  changed  and  bargaining  can  take  place. 

Discussions  must  be  meaningful  and  fair.  The  following  must  be  insured  when  conducting 
discussions: 

•  Identify  all  deficiencies  in  the  proposal 

•  Specify  all  deficiencies  to  the  offeror 

•  Provide  a  reasonable  time  for  revision 

•  Make  a  complete  record  of  the  discussions 

•  Hold  discussions  with  all  other  offerors  in  the  competitive  range  (AMC 
Pamphlet  715-3,  1998;  NFCTC,  1998) 
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12.0  Selection  Decision 

For  Federal  Acquisitions  the  final  award  decision  belongs  to  the  Source  Selection  Authority. 
According  to  AMC  Pamphlet  715-3,  the  source  selection  authority  must  ensure  the  selection 
decision; 

•  Is  based  on  a  comparative  analysis  of  the  proposals 

•  Is  consistent  with  stated  evaluation  factors  and  subfactors;  and 

•  When  tradeoffs  are  permitted,  consider  whether  or  not  perceived  benefits  are 
worth  any  price  premiums. 

•  Make  the  decision  on  a  rational  basis  and  set  it  forth  in  an  independent,  stand- 
alone defensible  document. 

The  minimum  information  that  is  needed  to  document  the  award  should  include: 

•  A  description  of  the  acquisition 

•  The  names  of  offerors 

•  A  summary  of  the  strengths  and  weaknesses  of  each  proposal  and  offeror 

•  The  reasons  why  the  selected  contractor  provides  the  greatest  probability  of 
satisfying  the  Government's  requirements.  (AMC  Pamphlet  715-3,  1998, 
NFCTC,  1998) 

The  below  table  is  the  AMC  Pamphlet  715-3's  Blueprint  for  the  source  selection  decision 

document. 

Documenting  the  Selection  Decision  and  Awarding  the  Contract 


Source  Selection  Decision  Document  for  (specify  product/service  &  RFP  #) 


1.  Decision  Statement. 


— ■ — ' ' - ■ — 

Example:  As  Source  Selection  Authority  for  this  acquisition,  I  have  determined  that  the  XYZ 

product/service  proposed  by  Offeror  C  provides  the  best  overall  value  to  satisfy  Army  needs. 

This  selection  was  made  based  upon  the  factors  and  subfactors  established  in  the  solicitation  and 

my  integrated  assessment  and  comparison  of  the  strengths,  weaknesses,  and  risks  of  the 

proposals  submitted  in  response  to  the  solicitation.  This  memorandum  documents  the  basis  for 

my  decision.    
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2.  Brief  description  of  the  product/service  called  for  in  the  solicitation. 


3.  Brief  description  of  the  Basis  for  Award  including  the  major  factors  against  which 
proposals  were  measured  and  their  relative  order  of  importance. 


4.  A  list  of  offerors  in  the  competitive  range. 


5.  Rationale  for  business  judgments  and  tradeoffs.  Include  the  following: 


•  Succinctly  compare  each  proposal  to  each  of  the  others,  focusing  on  key  proposal  differences 
(strengths,  weaknesses,  and  risks)  that  surfaced  in  the  evaluation  and  their  impact  on  the 
acquisition  objectives. 


Clearly  explain  specific  tradeoffs  that  led  to  the  decision. 


•  Clearly  explain  the  specific  benefits  of  technically  superior  offeror(s)  and  why  they  are  or  are 
not  significant  enough  to  warrant  any  additional  cost. 


6.  Summary. 


Example:  In  summary,  based  on  my  integrated  assessment  of  all  proposals  in  accordance  with 
the  specified  evaluation  factors  and  subfactors,  it  is  my  decision  that  Offeror  C's  proposal  offers 
the  best  overall  value. 


Signature 

Source  Selection  Authority 


13.0  Notification  and  Debriefing  of  Unsuccessful  Offerors 

The  FAR  15.503  requires  that  within  three  days  after  the  date  of  contract  award,  the 

contracting  shall  provide  written  notification  to  each  offeror  whose  proposal  was  in  the 

competitive  range  but  was  not  selected  for  award.  This  notice  must  include; 

(i)  The  number  of  offerors  solicited; 

(ii)  The  number  of  proposals  received; 

(iii)  The  name  and  address  of  each  offeror  receiving  an  award; 

(iv)  The  items,  quantities,  and  any  stated  unit  prices  of  each  award.  If  the  number  of 

items  or  other  factors  makes  listing  any  stated  unit  prices  impracticable  at  that  time, 

only  the  total  contract  price  need  be  furnished  in  the  notice.  However,  the  items, 

quantities,  and  any  stated  unit  prices  of  each  award  shall  be  made  publicly  available, 

upon  request;  and 

(v)  In  general  terms,  the  reason(s)  the  offeror's  proposal  was  not  accepted,  unless  the 

price  information  in  paragraph  (b)(l)(iv)  of  this  section  readily  reveals  the  reason.  In  no 

event  shall  an  offeror's  cost  breakdown,  profit,  overhead  rates,  trade  secrets, 

manufacturing  processes  and  techniques,  or  other  confidential  business  information  be 

disclosed  to  any  other  offeror. 
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It  is  my  experience  that  this  is  not  a  difficult  step,  however,  if  paragraph  (v)  is  done  with 
enough  detail,  it  may  help  contractors  understand  why  they  did  not  get  the  award  and  eliminate 
the  need  for  post  award  debriefings.  For  some  solicitations  we  edited  the  technical  evaluation 
report  and  sent  the  contractor  the  comments  from  the  Technical  evaluation  report  that  pertained 
to  their  own  proposal.  The  contractors  felt  that  this  information  was  very  useful  and  helpful 

The  Federal  Acquisition  Regulation  also  requires  post-award  debriefings.  These  debriefings 
are  usually  chaired  by  the  Contracting  Officer  and  supported  by  members  of  the  Selection  Team. 
The  debriefing  can  be  oral,  in  writing,  or  by  any  other  method  that  is  acceptable  to  the 
Contracting  Officer.  At  a  minimum  the  Federal  Acquisition  regulation  part  15.506  requires  that 
the  debriefing  an  offeror  who  responds  on  time  (within  three  days  of  written  notification  of 
contract  award)  shall  be  debriefed  and  furnished  the  basis  for  the  selection  decision  and  contract 
award.  It  also  states  that, 

(d)  At  a  minimum,  the  debriefing  information  shall  include— 

(1)  The  Government's  evaluation  of  the  significant  weaknesses  or  deficiencies  in  the 
offeror's  proposal,  if  applicable; 

(2)  The  overall  evaluated  cost  or  price  (including  unit  prices)  and  technical  rating,  if 
applicable,  of  the  successful  offeror  and  the  debriefed  offeror,  and  past  performance 
information  on  the  debriefed  offeror; 

(3)  The  overall  ranking  of  all  offerors,  when  any  ranking  was  developed  by  the 
agency  during  the  source  selection; 

(4)  A  summary  of  the  rationale  for  award; 

(5)  For  acquisitions  of  commercial  items,  the  make  and  model  of  the  item  to  be 
delivered  by  the  successful  offeror;  and 

(6)  Reasonable  responses  to  relevant  questions  about  whether  source  selection 
procedures  contained  in  the  solicitation,  applicable  regulations,  and  other  applicable 
authorities  were  followed. 

(e)  The  debriefing  shall  not  include  point-by-point  comparisons  of  the  debriefed 
offeror's  proposal  with  those  of  other  offerors.  Moreover,  the  debriefing  shall  not 
reveal  any  information  prohibited  from  disclosure  by  24.202  or  exempt  from  release 
under  the  Freedom  of  Information  Act  (5  U.S.C.  552)  including— 

(1)  Trade  secrets; 

(2)  Privileged  or  confidential  manufacturing  processes  and  techniques; 

(3)  Commercial  and  financial  information  that  is  privileged  or  confidential,  including 
cost  breakdowns,  profit,  indirect  cost  rates,  and  similar  information;  and 
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(4)  The  names  of  individuals  providing  reference  information  about  an  offeror's  past 

performance. 

(f)  An  official  summary  of  the  debriefing  shall  be  included  in  the  contract. 

It  is  my  experience  that  during  debriefings  the  contractors  want  to  know  how  they  can  do 
better  and  sometimes  what  the  other  contractors  did  to  score  higher  than  them    It  is  my  opinion 
that  it  is  important  for  the  Government  to  divulge  as  much  information  as  possible  to  help  a 
contractor  improve  their  proposal,  but  it  is  imperative  that  the  Government  representatives  stay 
within  the  bounds  of  the  offeror's  own  proposal.  For  instance,  one  of  the  evaluation  factors  we 
often  used  was  something  like  how  will  you  manage  quality  on  the  project.  Some  contractors 
took  an  approach  of  describing  their  quality  control  organization  and  their  companies 
commitment  to  quality  and  how  they  manage  quality  overall  and  by  doing  this  they  would  show 
that  they  knew  how  to  manage  quality  on  any  project.  This  is  fine  and  was  usually  graded 
"Acceptable"  unless  they  really  could  show  a  stand  out  quality  organization.  Another  contractor 
may  have  gone  into  their  overall  organization  and  quality  control  procedures  briefly  but  then 
really  focussed  on  how  they  would  insure  quality  on  specific  aspects  of  the  project  and  be  rated 
"Highly  Acceptable".  During  a  debriefing  the  offeror  who  was  rated  "acceptable"  would  want  to 
know  how  he  could  improve  his  rating.  The  government  representative  may  be  able  to  tell  him 
to  be  more  project  specific  next  time  but  can't  really  say  we  want  you  to  do  it  like  offeror  B. 

On  the  other  hand,  if  an  offer  responded  to  an  evaluation  factor,  "demonstrate  your  companies 
commitment  to  safety"  and  the  offeror  stated  they  have  a  safety  incentive  program  but  failed  to 
describe  it  and  received  an  "Acceptable"  rating.  The  Government  evaluator  can  tell  them  they 
could  improve  their  proposal  by  describing  their  incentive  program  in  more  detail  and  that  just 
stating  that  you  have  an  incentive  plan  does  not  merit  a  high  grade  unless  you  can  show  its 
effectiveness  and/or  value  to  the  employees. 
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Mr.  Richard  Palmer  of  "C"  Construction  Company,  Inc.,  said  that  he  would  very  much  like  it 
if  the  Government  could  provide  more  information  on  debriefings  and  maybe  at  least  show  the 
"grades"  or  ratings  other  contractors  received.  He  said  that  it  was  his  opinion  that  there  is  some 
perception  in  the  industry  that  sometimes  the  Government  Agency  has  a  contractor  in  mind  and 
that  contractor  is  given  some  partial  treatment.  He  felt  that  if  all  of  the  "ratings"  were  shown  it 
might  alleviate  that  perception  somewhat  and  help  contractors  gauge  how  they  are  doing 
compared  to  other  contractors. 

Ms.  Claudia  Hunley,  ACOE,  stated  that  debriefings  were  very  valuable  for  both  the 
Government  and  the  contractors  and  that  the  Government  was  able  to  learn  how  to  make  it  easier 
to  prepare  and  evaluate  proposals 
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Part  III 
Summary 


14.0  Contractors  View 


The  overall  views  of  the  contractors  that  I  interviewed  were  consistent.  Most  of  the 
contractors  felt  that  the  process  was  fair  but  some  felt  that  the  personalities  and/or  agency  could 
and  sometimes  did  bias  the  system.  The  major  points  from  the  contractors  were: 

•  High  Cost  -  Preparing  proposals,  especially  Design-Build,  is  expensive.  The  range  of 
rough  cost  I  received  was  from  $25K  for  smaller  jobs  to  $100K  for  the  more  complex 
solicitations  (Garcia,  Richard;  Pepper,  David).  Mr.  David  Pepper  of  Harry  Pepper  & 
Associates,  Inc.  said  that  he  felt  the  high  cost  made  it  difficult  for  smaller  contractors  to 
compete. 

It  is  worth  noting  that  the  Florida  DOT  provides  stipends  for  the  contractors  that  are  in 
the  2nd  Phase  of  the  2  Phase  Source  Selection  Process  on  Design  Build  projects  to  help 
offset  the  cost  of  preparation  of  the  proposals.  Mr.  David  Pepper  and  Mr.  Jerry  Stanley  of 
Hughet  Construction  said  that  not  getting  a  stipend  definitely  discourages  potential  offerors 
from  competing  on  certain  projects. 

•  Better  coordination  between  owner  and  end  user/facility  tenant  (Specifically  Design- 
Build)  -  Both  Mr.  Richard  Garcia  and  Mr.  David  Pepper  stated  that  the  end  user/tenant 
does  not  really  have  a  clear  picture  of  what  the  end  product  is.  Once  the  contract  is 
awarded  and  the  final  designs  and/or  construction  starts  the  tenant  finds  that  they  thought 
they  were  getting  something  else,  usually  something  better,  than  what  was  actually  in  the 
contract    Additionally,  Mr.  David  Pepper  said  that  he  much  preferred  the  Design-Bid- 
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Build  process  partly  because  the  A&E  does  not  have  the  face  to  face  time  with  the  end 
users  when  developing  the  plans  and  specifications. 

It  is  interesting  to  note,  as  stated  in  section  5.0,  that  the  end  users  are  included  in  all  of 
the  agencies/organizations  selection  teams.  It  is  my  experience  in  the  Navy  that  this 
problem  is  inherent  in  the  planning,  design  and/or  development  of  the  scope  of  work,  and 
that  the  perceived  lack  of  coordination  is  caused  by  many  factors  outside  the  scope  of  this 
paper. 

•  Evaluation  Factors/Criteria  is  reasonable  -  All  of  the  contractors  felt  that  the  evaluation 
factors/criteria  have  been  developed  and  are  reasonable  and  did  not  feel  that  there  were  any 
major  problems  with  them.  However,  Mr.  Knoll  of  J.  A.  Jones  Management  Services  did 
say  that  he  felt  if  there  were  too  many  factors/criteria  and  subfactors  that  the  important 
discriminators  in  a  solicitation  were  lost.  I  specifically  expected  contractors  to  say  that 
they  felt  the  some  factors  were  overly  burdensome  and  didn't  understand  what  the 
agency/organization  needed   However  this  was  not  the  case.  It  is  apparent  that  the 
agencies  that  I  interviewed  have  fairly  quickly  learned  what  factors/criteria  work  well  and 
don't  work  well. 

•  Criteria  and/or  Proposal  strengths  not  enforced  -  Mr.  Richard  Palmer  of  "C"  Construction 
Co.  Inc.  said  that  the  design  criteria  needed  to  be  more  prescriptive  and/or  better  defined. 
He  felt  that  sometimes  the  government  would  require  something  and  than  later  relaxes  the 
criteria.  For  example,  the  government  may  specify  some  minimum  sound  rating  criteria 
and  than  accept  a  proposal  that  really  did  not  meet  the  minimum  sound  rating  criteria.  Mr. 
Knoll  stated  that  he  felt  some  contractors  strategy  was  to  tell  the  government  what  they 
want  to  hear  to  win  but  then  not  actually  follow  through  and  perform  the  work  they 

45 


promised  in  their  proposals.  He  also  said  that  sometimes  the  proposals  would  be  evaluated 
favorably  for  things  that  once  the  contract  was  awarded  and  execution  started,  the  post 
award  side  of  the  contract  administration  did  not  feel  was  important 

•    Other  General  comments  - 

•  Don't  require  color  renderings  they  are  very  expensive 

•  Don't  change  the  format  of  required  information  from  one  solicitation  to  the  next. 

Mr.  Palmer,  said  that  one  agency  would  ask  for  Past  performance  in  one  prescribed  way 
or  on  a  certain  form  for  one  solicitation  and  then  ask  for  the  same  information  in  a  different 
format  on  the  next  solicitation.    He  felt  it  would  be  much  better  if  the  government  would 
be  specific  about  what  they  wanted  and  leave  the  format  up  to  the  individual  contractors. 

•  Don't  require  the  same  information  to  be  submitted  twice. 

For  example,  requiring  information  on  Past  performance  in  the  pre-qualification  stage  and 
then  requiring  the  same  information  again  from  the  contractors  on  the  short  list. 

•  Debriefings  are  very  useful  and  helpful. 

•  Increased  use  of  Oral  Presentations  would  be  beneficial  to  help  insure  that  contractors 
understand  requirements 

15.0  Conclusions  and  Recommendations 

The  Source  Selection  Process  continues  to  gain  popularity  and  is  seeing  increased  use.  All  of 
the  owner  representatives  that  I  talked  to  felt  that  it  helped  decrease  adversarial  relationships  and 
improve  the  quality  of  construction/service.  In  fact,  Mr.  Larry  Fisk  (SOUTHDIV)  said  their 
office  has  almost  completely  abandoned  the  IFB  processes  because  the  acquisition/pre-award 
time  is  not  that  much  different  from  Source  Selection  and  the  dividends  in  performance  are 
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worth  the  extra  time.  He  said  that  an  average  IFB  takes  61  days  to  award  as  compared  to  75  days 
for  simple  source  selections. 

The  process  is  still  continuing  to  develop  and  improve.  Different  Agencies/organizations 
have  different  policies  and  procedures  and  have  different  levels  of  experience  with  the  process. 
From  my  interviews  experience  seems  to  bring  about  the  biggest  process  improvements  and 
almost  everyone  I  talked  to  said  that  at  first  they  asked  for  too  much  detail  and/or  asked  for 
information  that  did  not  discriminate  between  offerors. 

Arguably  the  most  important  step  in  the  process  is  the  selection  and  development  of 
evaluation  factors.  If  the  evaluation  factors  are  clear  enough  and  as  simple  as  possible  without, 
of  course,  hampering  offeror's  innovation  and  creativity  the  selection  of  the  "Best  Value" 
contractor  will  be  much  easier.  If  the  evaluation  factors  are  unclear  and/or  complex  there  is  a 
greater  risk  of  the  evaluation  getting  bogged  down  and  taking  too  much  time,  confusion  amongst 
offerors  and  evaluators,  and  contractor  protests. 

The  second  most  important  step  is  the  evaluation  of  the  proposals  itself.  Even  if  the 
evaluation  factors  are  complex  and/or  unclear  if  the  evaluation  team  rates  the  proposals 
according  to  the  evaluation  factors  and  rate  them  consistently  the  conclusions  and 
recommendations  will  stand  in  the  face  of  a  GAO  protest  or  Federal  Court  proceedings.  For  this 
reason  the  Technical  Evaluation  Team  should  have  some  training  so  they  understand  the  rules 
and  guidelines  they  must  follow  and  their  importance  in  the  source  selection  process. 

I  believe  that  the  Source  Selection  process  is  the  best  available  way  to  get  the  best  value 
possible  for  Government  construction  projects    It  allows  for  the  use  of  non-price  related  factors 
that  are  not  available  with  the  IFB  process    It  helps  to  force  contractors  who  want  to  continue  to 
win  contracts  to  care  about  their  performance  ratings  on  projects,  helps  reduce  the  adversarial 
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relationship  between  the  Government  and  contractors  and  in  the  end  results  in  an  overall  higher 
quality  project  that  is  much  easier  to  administer 
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APPENDIX  A 


Guidelines  for  a  Source  Selection  Plan 

from 

AMC  Pamphlet  715-3 

Contracting  for  Best  Value 

A  Best  Practices  Guide  to  Source  Selections 


Source  Selection/Evaluation  Plan 


Section  I.  Summary  Description 

A.  Item/Service  to  be  Acquired 

B.  Proposed  Contract  Type  (Choose  one) 

_FFP 

_  FP  with  Incentives 
_Cost 
Cost  with  Incentives 

_  Hybrid  (Explain) 

C.  Source  Selection  Process  (Choose  one) 

_  Tradeoff 

_  Low  Price  Technically  Acceptable 
Hybrid  (explain) 

D.  Special  Evaluation  Techniques  (explain,  if  applicable) 
e.g.  oral  presentations,  multi-step  techniques 


Section  n.  Source  Selection  Team 

A.  Members.  List  the  evaluation  team  members  by  name  and  functional  area  (including 
advisors,  if  applicable). 

B.  Team  Member  Certifications:  Verify/attach  by  reference  certificates  from  each  team 
member  protecting  unauthorized  release  of  source  selection  or  proprietary  information. 


Section  HI.  Evaluation  Factors  and  Subfactors 

A.  Proposal  Evaluation  Information.  State  all  factors,  subfactors  and  their  relative  order  of 
importance  exactly  as  they  will  appear  in  the  solicitation.  If  you  elect  to  include  desirable 
objectives  or  features  you  would  be  willing  to  pay  extra  for,  include  an  explanation  of  how  they 
will  be  evaluated  and  whether  or  not  credit  will  be  given  for  exceeding  such  desirables. 

B.  Proposal  Submission  Information.  State  the  instructions  to  offerors  for  preparing  and 
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submitting  proposals  exactly  as  they  will  appear  in  the  solicitation. 


Section  IV.  Description  of  Evaluation  Procedures 

A.  Rating  System.  Describe  the  evaluation  technique  (ratings  and  their  definitions)  that  will  be 
used  to  rate  the  proposals. 

B.  Evaluation  Standards.  Describe  the  standards  or  target  levels  you  will  use  to  measure  how 
well  a  proposal  addresses  each  evaluation  factor  and  subfactor. 

C.  Procedures  for  Recording  the  Evaluation 

e.g.  Worksheet(s)  for  recording  evaluator  ratings  with  supporting  narrative 

D.  Schedule.  Identify  the  schedule  for  key  source  selection  activities. 


Section  V.  Rationale  for  the  Evaluation  Scheme 


A.  Factors  and  Subfactors.  Explain  how  the  selected  factors  and  subfactors  reflect  the 
circumstances  of  your  particular  acquisition  and  the  results  of  market  research  and  other 
presolicitation  exchanges  with  industry. 


B.  Weights.  Explain  how  the  relative  importance  of  the  factors  and  subfactors  reflect  the 
acquisition's  primary  objective.  Explain  why  the  most  heavily  weighted  factors  are  the  "drivers" 
of  the  source  selection. 
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APPENDIX  B 

GAO  CASE  DECISIONS 


BNUMBER:   B-280853 

DATE:   November  24,  1998 

TITLE:  Durocher  Bock  &  Dredge/Black  &  Veatch,  A  Joint  Venture,  B- 

280853,  November  24,  1998 

***•  +  **  +  +  *****  +  *•****•******  +  *•*•*•*****  +  **********•*******•***  +  +  *•*•*■** 

DOCUMENT  FOR  PUBLIC  RELEASE 

The  decision  issued  on  the  date  below  was  subject  to  a  GAO  Protective 

Order.   This  redacted  version  has  been  approved  for  public  release. 

Matter  of: Durocher  Dock  &    Dredge/Black  &  Veatch,  A  Joint  Venture 

File:B-280853 

Date:November  24,  1998 

Peter  J.  Ippolito,  Esq.,  Hillyer  &  Irwin,  for  the  protester. 

Mark  Christopher,  Esq.,  Lis  B.  Young,  Esq.,  and  George  N.  Brezna, 

Esq.,  Naval  Facilities  Engineering  Command,  for  the  agency. 

Aldo  A.  Benejam,  Esq.,  and  Christine  S.  Melody,  Esq.,  Office  of  the 

General  Counsel,  GAO,  participated  in  the  preparation  of  the  decision. 

DIGEST 

1.  Allegation  that  agency  improperly  evaluated  protester's  proposal 
is  denied  where  the  record  shows  that  the  agency  evaluated  the 
proposal  in  accordance  with  the  evaluation  factors  announced  in  the 
solicitation  and  record  reasonably  supports  protester's  overall  lower 
technical  rating. 

2.  Disparities  in  evaluation  ratings  among  technical  evaluators  do 
not  establish  that  the  evaluation  process  was  flawed  or  not  rationally 
based  in  view  of  the  potential  for  disparate  subjective  judgments  of 
different  evaluators  on  the  relative  strengths  and  weaknesses  of 
technical  proposals. 

3.  Allegations  by  protester  whose  phase  one  proposal  was  reasonably 
ranked  eighth,  that  agency  improperly  evaluated  the  first-  and 
fourth-ranked  proposals  is  dismissed  where,  even  if  protester's 
allegations  were  sustained,  protester's  proposal  would  not  be  eligible 
to  proceed  to  phase  two  of  the  procurement;  protester  is  not  an 
interested  party  to  pursue  this  aspect  of  the  protest. 

DECISION 

Durocher  Dock  &  Dredge/Black  &  Veatch,  A  Joint  Venture  protests  the 
rejection  of  its  proposal  under  request  for  proposals  (RFP)  No. 
N62467-98-R-0968,  issued  by  the  Depailmenl  of  Lhe  Navy  Lo  impiove 
Wharf  D,  at  the  Naval  Station  in  Mayport,  Florida.   The  protester 
argues  that  the  evaluation  of  its  proposal  was  inconsistent  with  the 
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solicitation. 

We  deny  the  protest. 

The  Navy  issued  the  RFP  on  May  20,  1998,  as  phase  one  of  a  two-phase 
procurement  under  the  procedures  set  out  at  Federal  Acquisition 
Regulation  Subpart  36.3.   RFP   sec.   00202,  amend.  No.  0001,  part  1.1.2 
The  RFP  is  for  the  design  and  construction  of  a  major  waterfront 
improvement  project  at  the  Naval  Station,  Mayport,  Florida.   The 
solicitation  advised  offerors  that  the  agency  would  evaluate  proposals 
in  two  phases.   Under  phase  one,  at  issue  here,  the  RFP  listed  the 
following  evaluation  factors  in  descending  order  of  importance:   past 
performance,  technical  qualifications,  management  approach,  and  small 
business  subcontracting  effort.   Id.   Based  upon  the  results  of  that 
evaluation,  the  Navy  would  then  select  a  maximum  of  five  of  the  "most 
highly-qualified  offerors"  to  submit  phase  two  proposals,  which  are  to 
be  evaluated  based  on  technical  considerations  and  price.   Award  is  to 
be  made  to  the  offeror  whose  proposal  is  deemed  to  represent  the  best 
value  to  the  government,  considering  price  and  technical  factors.   Id. 

Twelve  firms,  including  the  protester,  responded  to  the  RFP  by  the 
time  set  on   June  19,  for  receipt  of  phase  one  proposals.   A  technical 
evaluation  board  (TEB)  evaluated  proposals  by  assigning  adjectival 
ratings  (exceptional,  acceptable,  marginal,  or  unacceptable)  under 
each  evaluation  factor,  and  an  overall  proposal  rating.   Based  on  the 
overall  ratings,  the  TEB  then  ranked  proposals  from  the  most 
highly-rated  (exceptional)  to  the  lowest-rated  (unacceptable)  as 
follows : 


[DELETED]  Exceptional 

[DELETED]  Exceptional 

Offeror  A  Acceptable 

[DELETED]  Acceptable 

Offeror  B  Acceptable 

Offeror  C  Marginal 

Offeror  D  Marginal 

Durocher  Dock  &  Dredge/Black  &  VeatchMarginal 

Offeror  E  Marginal 

Offeror  F  Marginal 

Offeror  G  Marginal 

Offeror  H  Unacceptable 

Agency  Report  at  2 . 

Based  on  the  results  of  the  phase  one  evaluation,  the  TEB  found  that 
the  five  highest-ranked  proposals  had  demonstrated  superior  past 
performance  and  technical  qualifications  over  the  remaining  seven 
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firms,  and  recommended  to  the  source  selection  board  (SSB)  that  the 
firms  that  submitted  those  proposals  advance  to  phase  two  of  the 
procurement.   The  SSB  accepted  that  recommendation  and  by  letters 
dated  July  16,  the  agency  informed  all  offerors  whether  they  were 
selected  to  proceed  to  phase  two.   Following  a  debriefing  by  the  Navy 
conducted  on  July  29,  the  protester  filed  an  agency-level  protest, 
which  the  Navy  denied.   This  protest  to  our  Office  followed. 

PROTESTER'S  CONTENTIONS 

The  protester  argues  that  the  evaluation  of  its  proposal  was  flawed 
and  inconsistent  with  the  RFP  in  several  respects.   For  instance, 
Durocher  argues  that  the  TF.R  deviated  from  the  evaluation  scheme 
announced  in  the  solicitation  by  emphasizing  certain  aspects  of  the 
work  described  in  the  solicitation  (primarily  electrical  work) ,  but 
that  the  Navy  failed  to  notify  offerors  that  this  portion  of  the  work 
would  be  considered  more  important  in  the  evaluation.   The  protester 
also  maintains  that  individual  evaluators  were  not  consistent  in 
applying  the  RFP's  evaluation  criteria.   Durocher  further  argues  that 
the  agency  improperly  evaluated  the  proposals  submitted  by  two  firms 
selected  to  proceed  to  phase  two  of  the  procurement. 

DISCUSSION 

Our  Office  will  not  engage  in  an  independent  evaluation  of  proposals 

nor  make  an  independent  determination  of  their  relative  merits. 

Litton  Sys.,  Inc.,  B-239123,    Aug.  7,  1990,  90-2  CPD   para.   114  at  9. 

Rather,  we  review  the  agency's  evaluation  only  to  ensure  that  it  was 

reasonable  and  consistent  with  applicable  statutes  and  regulations  as 

well  as  with  the  terms  of  the  solicitation.   Sensis  Corp.,  B-265790.2, 

Jan.  17,  1996,  96-1  CPD  para.   77  at  6.   A  protester's  mere  disagreement 

with  the  agency's  conclusions  does  not  render  the  evaluation 

unreasonable.   ESCO,  Inc.,  B-225565,  Apr.  29,  1987,  87-1  CPD  para.   450  at 

7.   Based  on  our  review  of  the  record,  including  the  TEB's  narrative 

in  support  of  its  evaluation,  we  conclude  that  the  factor  and  overall 

ratings  assigned  the  protester's  proposal  are  reasonably  supported. 

Below  we  discuss  a  representative  sample  of  the  TEB's  findings  with 

respect  to  the  protester's  proposal  in  support  of  our  conclusion. 

Past  Performance 

The  TEB  assigned  Durocher ' s  proposal  a  rating  under  this  factor  of 
"acceptable  minus"  (A-),[l]  indicating  the  consensus  of  the  TEB  that 
the  proposal  was  acceptable,  but  contained  some  weaknesses  in  this 
area.   Under  both  subf actors  in  this  area — design  team  and 
construction  team--of ferors  were  required  to  submit  past  performance 
narratives  for  up  to  three  projects  that  demonstrated  experience  in 
performing  work  similar  to  that  described  in  part  1.1.3  of  section 
00202  of  the  RFP,  the  Specialized  Project  Requirements . [2 ]   Offerors 
were  also  instructed  to  describe  up  to  three  partnering/teaming 
arrangements  with  design  and  construction  teams. 

The  TEB  identified  several  strengths  in  the  protester's  proposal  under 
both  subfactors  in  this  area.   For  instance,  the  TEB  found  that  all 
three  project  narratives  Durocher  submitted  demonstrated  work  relevant 
to  the  Wharf  D  improvement  project  in  that  they  all  were  major- 
waterfront  projects,  and  their  scope  included  most  or  all  of  the  work 
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elements  required  to  accomplish  this  project.   The  TEB  identified 
several  weaknesses,  however.   For  example,  the  TEB  found  that  the 
projects  Durocher  submitted  for  the  design  and  construction  teams  did 
not  contain  medium  voltage  electrical  distribution  work  (as  described 
in  part  1.1.3  of  section  00202  of  the  RFP) .   The  TEB  also  found  that 
Durocher 's  proposed  design  and  construction  teams  demonstrated  no  past 
performance  for  steam  systems,  and  had  minimal  experience  with  fueling 
systems.   One  evaluator  specifically  noted  that  the  proposed  design 
team's  projects  did  not  indicate  any  steam  system  design  experience, 
and  that  it  was  unclear  from  the  proposal  whether  the  designers  had 
performed  any  work  related  to  fuel  systems  on  any  of  the  three 
projects  Durocher  described  in  its  proposal.   Technical  Evaluation 
Worksheets.   The  TEB  further  found  that  none  of  the  key  personnel  in 
Durocher' s  proposed  design  team  had  worked  on  any  of  the  partnering 
arrangements  the  firm  described  in  its  proposal.   In  our  view,  the 
TEB's  consensus  rating  of  A-  under  this  factor  reasonably  reflected 
the  evaluators'  concerns  that  Durocher' s  proposal  had  not  demonstrated 
experience  in  performing  work  similar  to  that  described  in  the 
Specialized  Project  Requirements  section  of  the  RFP. 

Durocher  argues  that  by  downgrading  its  proposal  for  failing  to  show 
that  the  projects  it  submitted  contained  medium  voltage  electrical 
distribution  work,  the  TEB  gave  this  weakness  greater  significance 
than  announced  in  the  RFP.   In  this  connection,  Durocher  argues  that 
the  TEB  improperly  emphasized  the  "voltage  power  distribution  system" 
aspect  of  the  work  over  other  aspects,  and  that  the  Navy  failed  to 
inform  offerors  that  this  work  element  was  significantly  more 
important  than  any  other  aspect  of  construction  or  design. 
Protester's  Comments,  Oct.  2,  1998,  at  2. 

The  protester's  argument  that  the  TEB  emphasized  one  element 
(electrical  distribution  system)  over  other  elements  of  the  work  is 
not  supported  by  the  record.   As  already  stated,  the  record  shows  that 
the  TEB  considered  Durocher ' s  proposal  acceptable  under  this  factor, 
but  came  to  a  consensus  regarding  the  several  weaknesses  noted  above. 
Since  the  TEB  concluded  that  the  projects  Durocher  submitted  did  not 
involve  medium  voltage  electrical  distribution  work--one  of  the 
elements  contained  in  part  1.1.3  of  section  00202  of  the  RFP— the  TEB 
reasonably  downgraded  Durocher ' s  proposal  slightly  under  this  factor 
to  a  rating  of  A- .   The  fact  that  the  TEB  found  that  this  element  was 
missiny  did  noL  mean  thai  il  was  given  more  importance  Lhan  olhers, 
and  based  on  our  review  of  the  evaluation,  we  have  no  basis  to  object 
to  the  TEB's  rating. 

The  protester  points  out  that  some  individual  evaluators  rated 
Durocher' s  proposal  as  exceptional  in  this  area,  while  other 
evaluators  downgraded  its  proposal  for  the  medium  voltage  power 
distribution  weakness.   According  to  Durocher,  this  variation  in 
individual  ratings  demonstrates  that  the  members  of  the  TEB  were  not 
uniform  in  applying  the  RFP ' s  evaluation  criteria. 

It  is  not  unusual,  however,  for  individual  evaluators  to  have 
disparate  judgments  regarding  the  relative  strengths  and  weaknesses  of 
a  technical  proposal.   Syscon  Servs.,  Inc.,  B-23564'7,  Sept.  21,  1989, 
89-2  CPD  para.   258  at  5.   Disparities  in  evaluator  ratings  do  nut 
establish  that  the  evaluation  process  was  flawed  or  otherwise  not 
reasonable.   U.S.  Defense  Sys.,  Inc.,  B-245006.2,  Dec.  13,  1991,  91-2 
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CPD  para.   541  at  4.   Contrary  to  the  protester's  arqument,  the  record 
shows  that  only  one  evaluator  rated  Durocher's  proposal  exceptional 
while  the  other  evaluators  rated  Durocher's  proposal  either  marginal 
or  acceptable  in  this  area,  and  that  their  narrative  comments  merely 
reflect  the  individual  evaluators'  subjective  judgments  regarding  the 
relative  strengths  and  weaknesses  of  the  proposal  in  this  area.   In 
any  case,  notwithstanding  the  different  individual  ratings,  the  TEB 
reached  a  consensus  in  this  area. 

Technical  Qualifications 

The  TEB  assigned  Durocher's  proposal  a  rating  of  marginal  under  this 
factor .   Under  the  two  subfactors  evaluated  in  this  area — design  team 
and  construction   team--of ferors  were  instructed  to  submit  key 
personnel  staffing  that  demonstrated  experience  in  the  elements 
included  in  the  Specialized  Project  Requirements  provision  of  the  RFP. 
Offerors  were  also  required  to  provide  qualifications  and  staffing 
capability  to  support  the  project. 

Under  the  design  team  subfactor,  the  TEB  found  that  only  one 
individual  Durocher  proposed  as  "key  personnel"  had  participated  in 
any  of  the  projects  Durocher  described  in  its  proposal  (and  even  that 
individual  had  participated  in  only  one  project  described  in  the 
proposal),  and  identified  this  as  a  weakness  in  the  proposal.   The  TEB 
further  found  that  the  proposed  key  personnel  demonstrated  no 
waterfront  electrical  distribution,  steam,  or  fuel  system  experience 
and  considered  this  a  weakness  in  the  proposal.   Under  the 
construction  team  subfactor,  the  TEB  found  that  the  proposed  key 
personnel  demonstrated  no  waterfront  electrical  distribution 
experience,  and  that  the  waterfront  experiences  listed  for  key 
structural  personnel  were  minimal  in  scope.   The  protester  does  not 
rebuL  any  of  the  TEB's  findings  in  this  regard. 

In  its  proposal,  Durocher  identified  the  firm  of  [DELETED]  as  a 
mechanical  team  member  and  [DELETED]  as  the  electrical  installation 
team  member.   The  TEB  noted,  however,  that  the  protester  did  not 
include  any  personnel  data  for  [DELETED] ,  and  found  that  the  proposal 
demonstrated  no  specific  diesel  fuel  or  steam  work  experience  by 
conslruclion  projecl  managers,  super  intexidents,  ox  quality  assurance 
personnel . 

In  its  comments  on  the  agency  report,  Durocher  argues  that  both  firms 
it  identified  in  its  proposal  as  (mechanical  and  electrical)  team 
members  have  previously  worked  on  Navy  projects,  including  Mayport. 
According  to  the  protester,  therefore,  the  Navy  should  have  been 
familiar  with  these  firms  based  on  their  performance  on  those 
projects . 

Durocher's  reliance  on  the  previous  projects  performed  by  its  proposed 
team  members  without  regard  to  the  specific  information  required  by 
the  RFP  is  misplaced.   A  procuring  agency's  technical  evaluation  is 
dependent  upon  the  information  furnished  in  the  offeror's  proposal. 
Computerized  Project  Management  Plus,  B-247063,  Apr.  28,  1992,  92-1 
CPD   para.   401  at  3.   An  agency  is  not  required  to  overlook  a  flawed 
proposal  on  the  basis  of  the  offeror's  prior  performance;  on  the 
contrary,  all  offerors  are  expected  to  demonstrate  their  capability  in 
their  proposals.   Pedus  Bldg.  Servs.,  Inc.,  B-257271.3  et  al.,  Mar.  8, 
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1995,  95-1  CPD  para.   135  at  4.   Consequently,  the  agency  reasonably  relied 
on  Durocher's  proposal  in  determining  the  identified  weaknesses  in  its 
qualifications  and  staffing  capability  to  support  the  project. [3] 
Since  Durocher  has  not  shown  that  the  evaluated  weaknesses  are 
unfounded  based  on  the  information  contained  in  its  proposal,  we  have 
no  basis  to  object  to  the  evaluation. [4] 

Based  on  our  review  of  the  evaluation  record,  including  the 
protester's  proposal  and  the  TEB's  individual  worksheets,  we  think 
that  the  agency  reasonably  concluded  that  Durocher's  proposal  omitted 
material  information  regarding  its  past  performance  and  failed  to 
demonstrate  its  technical  qualifications  as  required  by  the  RFP. [5] 
Under  these  circumstances,  we  have  no  basis  to  object  to  the  TEB's 
overall  rating  of  marginal  of  the  protester's  proposal. 

The  protester  also  argues  that  the  agency  improperly  evaluated  the 
proposals  submitted  by  two  firms  selected  to  proceed  to  phase  two  of 
the  procurement.   In  this  regard,  Durocher  maintains  that  [DELETED] 
"has  been  in  business  for  only  five  months  and  has  yet  to  successfully 
complete  a  project."   Protester's  July  29,  1998  letter  to  the  Navy  at 
1-2.   Durocher  reiterates  this  argument  in  its  comments  on  the  agency 
report,  maintaining  that  the  record  "reveals  [DELETED]  was  formed  in 
October  of  1997."   Protester's  Comments,  Oct.  2,  1998,  at  4. 

In  addition,  Durocher  asserts  that  [DELETED]  "has  lost  virtually  all 
of  its  management  personnel  that  .  .  .  were  responsible  for  the 
construction  and  management  of  all  projects  listed  in  their 
performance  record."   Letter  from  Protester  to  the  Navy  at  2  (July  29, 
1998) .   According  to  Durocher,  "it  is  obvious  that  a  number  of 
[DELETED]  key  employees  left  [DELETED]  and  became  employees  of  the 
newly  formed  [DELETED]  contractor  corporation  in  1997."   Protester's 
Comments,  Oct.  2,  1998,  at  5.   The  protester  thus  questions  the  higher 
ratings  assigned  the  proposals  submitted  by  [DELETED]  and  [DELETED] . 

Under  the  bid  protest  provisions  of  the  Competition  in  Contracting  Act 
of  1984,     31  U.S.C.A.   sec.   3551-3556  (West  Supp.  1998),  only  an 
"interested  party"  may  protest  a  federal  procurement.   That  is,  a 
protester  must  be  an  actual  or  prospective  offeror  whose  direct 
economic  interest  would  be  affected  by  the  award  of  a  contract  or  the 
failure  Lo  award  a  conlracL.   Bid  PxuLesL  ReyulaLions,  4  C.F.R.   sec. 
21.0(a)  (1998).   Determining  whether  a  party  is  interested  involves 
consideration  of  a  variety  of  factors,  including  the  nature  of  issues 
raised,  the  benefit  of  relief  sought  by  the  protester,  and  the  party's 
status  in  relation  to  the  procurement.   Black  Hills  Refuse  Serv., 
B-228470,  Feb.  16,  1988,  88-1  CPD   para.   151  at  2-3.   A  protester  is  not  an 
interested  party  where  it  would  not  be  in  line  for  contract  award  were 
its  protest  to  be  sustained.   ECS  Composites,  Inc.,  B-235849.2,  Jan. 
3,  1990,  90-1  CPD  para.   7  at  1. 

Based  on  their  overall  ratings,  [DELETED]  proposal  (exceptional)  was 
ranked  first,  [DELETED]  proposal  (acceptable)  was  ranked  fourth,  while 
Durocher's  proposal  (marginal)  was  ranked  eighth  overall.   Since  we 
conclude  that  the  evaluation  of  Durocher's  proposal  was  reasonable, 
even  assuming  that  [DELETED]  and  [DELETED]  proposals  were  downgraded 
so  as  to  be  eliminated  from  phase  one,  Durocher's  proposal's  relative 
ranking  would  rise  only  from  eighth  to  sixth  place.   Since  the  RFP 
stated  that  a  maximum  of  five  of  the  "most  highly  qualified  offerors" 
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would  advance  to  phase  two,  RFP   sec.   00202,  amendment  No.  0001,  part 
I. 1.2.,  Durocher  would  not  be  eligible  to  proceed  to  phase  two  of  the 
competition  (instead,  the  higher-rated  proposals  submitted  by  offerors 
C  and  D  would  now  be  among  the  most  highly-rated  proposals) . 

Where,  as  here,  there  are  intervening  offerors  with  a  greater  interest 
in  the  procurement  than  the  protester,  we  generally  consider  the 
protester's  interest  to  be  too  remote  to  qualify  the  protester  as  an 
interested  party.   Four  Seas  and  Seven  Winds  Travel,  Inc.,  B-244916, 
Nov.  15,  1991,  91-2  CPD  para.   463  at  4.   Since  nothing  in  Durocher' s 
protest  would  alter  the  ratings  of  the  intervening  offerors  (Offerors 
C  and  D) ,  Durocher  would  not  be  eligible  to  proceed  to  phase  two  of 
the  competition  even  if  its  protest  allegations  were  sustained; 
accordingly,  Durocher  is  not  an  interested  party  to  challenge  the 
evaluation  of  the  [DELETED]  and  [DELETED]  proposals. 

The  protest  is  denied. 

Comptroller  General 
of  the  United  States 

1.  The  TEB  added  a  plus,  a  minus,  or  no  designation  to  the  factor- 
ratings  based  on  the  strengths  and  weaknesses  of  each  factor.   To 
assist  in  determining  proposal  rankings,  the  TEB  also  added  a  plus,  a 
minus,  or  no  designation  to  the  overall  proposal  ratings. 

2.  Part  1.1.3  of  section  00202  of  the  RFP  describes  the  project, 
including  the  main  design  and  construction  elements  such  as  dredging, 
structural,  mechanical,  electrical,  and  civil  utilities. 

3.  We  recognize  that  in  some  cases,  where  an  offeror's  proposal  refers 
Lo  informalion  regarding  ils  pasl  performance  and  Lhe  informalion  is 
personally  known  to  the  evaluators,  that  information  must  be 
considered  in  the  evaluation.   See,  e.g.,  International  Bus.  Sys., 

Inc.,  B-275554,  March  3,  1997,  97-1  CPD  para.   114  at  5  ("some  information 
is  simply  too  close  at  hand  to  require  offerors  to  shoulder  the 
inequities  that  spring  from  an  agency's  failure  to  obtain,  and 
consider,  the  information").   That  is  not  the  case  here.   On  the 
contrary,  Durocher 's  proposal  merely  named  its  two  proposed  electrical 
and  mechanical  team  members,  and  generally  stated  that  their  selection 
was  based  on  their  "knowledge  of  the  Naval  Station  Mayport,  and 
experience  and  familiarity  with  government  contracting."   Durocher 's 
Proposal  at  13.   The  proposal  did  not  identify  any  Navy  contracts  at 
the  Mayport  facility  with  which  the  firms  were  involved  and  did  not 
describe  the  services  those  firms  rendered  in  connection  with  those 
contracts;  nor  does  the  proposal  identify  the  specific  activity  for 
which  the  work  was  performed. 

4.  Regarding  the  evaluation  of  [DELETED]  proposal,  one  of  the  firms 
which  was  retained  in  the  competition,  Durocher  argues  that  the  TEB 
improperly  considered  as  a  strength  information  regarding  a  proposed 
electrical  and  mechanical  subcontractor  that  was  apparently  not 
submitted  in  that  firm's  proposal.   In  this  regard,  the  TEB  found  as 
one  of  nine  sLrenylhs  under  Lhe  Pasl  Performance  facLor  Lhal, 
"[a]lthough  not  submitted,  .  .  .  [the]  (elec.  &  mech.  sub.)  has 
received  several  excellent  commendations  from  SOUTHDIV  for  relevant 
work."    Technical  Evaluation  Board  Report  at  13.   Even  assuming  that 


60 


the  TEB  should  not  have  considered  that  information,  given  the  other 
strengths  noted  and  the  superior  factor  ratings  assigned  the  proposal 
in  other  areas,  there  is  no  reason  to  conclude  that  the  "exceptional" 
rating  of  [DELETED]  proposal,  or  the  overall  proposal  rating  of 
"exceptional  minus,"  would  change. 

5.  The  TEB  assigned  Durocher's  proposal  a  rating  of  "marginal  plus" 
under  the  third  evaluation  factor,  management  approach.   Offerors  were 
required  to  submit  an  organizational  chart  for  design  and  construction 
teams  demonstrating  contractual  arrangements  and  lines  of  authority 
between  key  personnel.   RFP,  amend.  No.  0001,  part  1.1.5,  factor  C. 
The  TEB  found  that  while  the  organizational  charts  Durocher  included 
in  its  proposal  listed  all  design  and  construction  key  personnel,  the 
charts  and  matrices  presented  were  unclear.   Also,  the  TEB  concluded 
that  the  narrative  included  with  the  charts  did  not  demonstrate  an 
adequate  understanding  of  the  design-build  process.   Based  on  our 
review  of  the  charts,  the  accompanying  explanation,  and  the  TEB's 
narrative  in  support  of  its  evaluation,  and  since  Durocher  has  not 
rebutted  any  of  the  evaluators'  findings,  we  think  the  TEB's  rating  in 
this  area  is  reasonable. 
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DIGEST 

1.  Agency's  best  value  award  selection  is  not  reasonably  supported 
where  the  record  reflects  that  the  selection  of  the  awardee  was  based 
on  an  unequal  evaluation  of  the  protester's  and  awardee 's  proposals. 

2.  Agency  improperly  waived  definitive  responsibility  criterion  which 
required  the  awardee  possess  a  Virginia  Real  Estate  Broker  License  at 
the  time  of  award,  where  the  agency  determined  the  awardee  to  be 
responsible,  despite  not  possessing  the  license. 

DECISION 

Tidewater  Homes  Realty,  Inc.  protests  the  award  of  a  contract  to 
CitiWest  under  request  for  proposals  (RFP)  No.  H03R95062400000,  a 
total  small  business  set-aside,  issued  by  the  United  States  Department 
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of  Housing  and  Urban  Development  (HUD) ,  for  real  estate  asset  manager 
(REAM)  services  for  single-family  properties  owned  by  HUD  or  in  its 
custody  in  Virginia  Beach,  Virginia  area  under  a  firm,  fixed-price, 
indefinite  quantity,  contract  for  a  base  year  with  4  option  years. 

We  sustain  the  protest. 

The  RFP,  issued  as  a  small  business  set-aside  on  June  1,  1995, 
provided  for  a  best  value  award  with  the  technical  evaluation  being 
worth  more  than  cost/price.   The  RFP  listed  the  following  technical 
evaluation  factors  and  corresponding  values: 

1.  Demonstrated  experience  in  the  management  of  single-family 
properties  similar  to  and  in  a  like  area  as  those  covered  by  this 
solicitation.   [25  points] 

2.  Demonstrated  experience  in  developing  lists  of  needed 
repairs,  such  as  is  required  by  HUD's  Minimum  Properly  Standards 
(MPS),  and  estimating  the  cost  of  repairs.   [25  points], 

3.  Demonstrated  experience  in  soliciting  repair  bids, 
coordination  and  overseeing  repair  work,  and  inspecting  for 
satisfactory  work  completion.   [15  points] 

4 .  Demonstrated  experience  in  managing  a  rental  program, 
including  establishing  lair  market  rentals  and  collections  from 
present  and  former  tenants,  for  single-family  properties.   [10 
points] 

5.  Understanding  of  HUD  objectives  and  the  required  tasks  as 
specified  in  the  solicitation.   [10  points] 

6.  Evidence  of  adequate  of f ice--staf f ed  with  appropriately 
trained  staff  and  equipped  appropriately  (or  the  ability  to 
establish  such) ,  reasonably  located  so  as  to  provide  convenient 
service  to  HUD  and  its  clients  in  the  area  to  be  served,  and  to 
carry  out  all  duties  specified  in  the  solicitation.   [15  points] 

As  part  of  their  technical  proposal,  offerors  were  required  to  submit 
a  completed  Form  477,  List  of  Repairs  (included  in  RFP) ,  for  a 
specified  property  in  Virginia  Beach.   Offerors  were  cautioned  that: 

"Proposals  submitted  without  this  form  will  not  be  disqualified 
from  competing  for  the  award,  but  omission  of  the  form  may 
adversely  affect  the  offeror's  technical  points  achieved  in 
[factor  2] ." 

In  addition  the  RFP  required: 

"The  contractor  must  supply  with  its  Technical  Proposal  evidence 
of  its  Virginia  Real  Estate  Broker  License--in  the  contractor's 
name  as  it  appears  on  the  offer--to  be  determined  responsible  and 
eligible  for  award."   [Emphasis  in  original.] 

At  the  pre-proposal  conference,  a  HUD  official  stated  that  if  evidence 
of  the  foregoing  license  were  not  included  in  the  proposal,  "evidence 
of  [the  offeror's]  ability  to  provide  the  license  at  award  must  be 


62 


provided."   The  RFP  expressly  stated  that  proposals  would  be  initially 
screened  to  ensure  that  they  contained  a  Form  477  for  the  specified 
property  and  evidence  of  a  Virginia  Real  Estate  Broker  License. 

HUD  received  12  initial  proposals  by  November  13.   Tidewater,  whose 
principals  were  a  manager  and  former  employee  of  the  local  incumbent 
contractor  performing  these  services,  and  CitiWest,  from  Tucson, 
Arizona,  both  submitted  proposals.   A  technical  evaluation  panel 
(TEP) ,  comprised  of  members  from  the  regional  contracting  office  in 
Philadelphia,  Pennsylvania  evaluated  proposals.   Five  proposals, 
including  those  of  Tidewater  and  CitiWest,  were  included  in  the 
competitive  range.   Three  of  the  proposals,  including  CitiWest 's,  with 
a  score  of  91  points,  were  found  technically  acceptable  and  two, 
including  Tidewater's,  with  a  score  of  37  points,  were  found  capable 
of  being  made  acceptable. 

By  letter  of  June  21,  1996,  HUD  conducted  discussions  with  the 
competitive  range  offerors,  during  which  it  advised  Tidewater  of  the 
weaknesses  and/or  deficiencies  in  its  proposal.   HUD  received  best  and 
final  offers  (BAFO)  by  July  9.   CitiWest 's  BAFO  at  an  evaluated  unit 
price  of  $1,317  received  the  same  technical  score  of  91  points. 
Tidewater's  BAFO  at  an  evaluated  unit  price  of  $1,233  received  a 
technical  score  of  47  points. 

The  TEP  rated  Tidewater's  BAFO  relatively  low  under  five  of  the  six 
evaluation  factors,  finding  that  Tidewater's  proposal  lacked  pertinent 
detail  in  many  respects.   In  contrast,  CitiWest 's  proposal  received 
high  point  scores  under  all  of  the  factors  and  was  determined  to  be 
reasonably  priced.   HUD  determined  that  CitiWest 's  technically 
superior  proposal  was  worth  the  additional  cosL  because  iL  would 
likely  have  fewer  performance  problems,  resulting  in  less  costs  to  the 
government,  and  made  award  to  that  firm  on  August  23.   This  protest 
followed. 

Tidewater  protests  that  its  proposal  contained  the  required  pertinent 
details  and  was  misevaluated.   Tidewater  claims  that  the 
Philadelphia-based  TEP  was  biased  against  Tidewater,  as  well  as  other 
locally  based  companies,  and  that  HUD  may  have  harbored  resentment 
against  Tidewater  due  to  actions  that  Tidewater  undertook  in  a  prior 
procurement..   Tidewater  notes  that  it  submitted  essentially  the  same 
proposals  in  response  to  HUD  procurements  for  similar  REAM  services 
that  a  HUD  Richmond-based  TEP  rated  very  favorably.   Tidewater  also 
claims  that  CitiWest  did  not  have  the  Virginia  Real  Estate  Broker 
License  in  its  name  at  the  time  of  award  as  required  by  the  RFP. 

The  evaluation  of  proposals  is  a  matter  within  the  discretion  of  the 
contracting  agency.   Our  Office  will  only  question  the  agency's 
evaluation  where  it  lacks  a  reasonable  basis  or  conflicts  with  the 
stated  evaluation  criteria  for  award.   SC&A,  Inc.,  B-270160.2,  Apr. 
10,  1996,  96-1  CPD  para.   197.   The  record  must  reasonably  support  the 
evaluation  of  the  proposals,  Intown  Properties,  Inc.,  B-262236.2; 
B-262237.2,  Jan.  18,  1996,  96-1  CPD  para.   89,  and  it  is  fundamental  that 
the  contracting  agency  must  treat  all  offerors  equally;  it  must 
even-handediy  evaluate  offers  against  common  requirements  and 
evaluation  criteria.   Sci-Tec  Gauging,  Inc.;  Sarasota  Measurements  & 
Controls,  Inc.,  B-252406;  B-252406.2,  June  25,  1993,  93-1  CPD  494; 
Secure  Servs.  Technology,  Inc.,  B-238059,  Apr.  25,  1990,  90-1  CPD  para. 
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421.   As  illustrated  by  the  examples  below,  our  review  of  the  record 
reveals  that  CitiWest's  and  Tidewater's  proposal  were  unequally 
evaluated  under  the  RFP's  evaluation  factors. [1] 

For  example,  under  Factor  2,  Tidewater's  proposal  received  6  points, 
while  CitiWest's  proposal  received  24  points.   Tidewater's  low  score 
was  attributed  to  an  asserted  lack  of  detail  and  clarity  contained  in 
the  submitted  Form  477  on  the  Virginia  Beach  property  specified  in  the 
RFP  and  questions  regarding  Tidewater's  ability  to  prepare  cost 
estimates.   These  evaluated  problems  were  brought  to  Tidewater's 
attention  during  discussions.   In  its  BAFO,  Tidewater  basically 
explained,  with  some  further  elaboration,  why  it  believed  the 
submitted  Form  477  and  repair  list  satisfied  the  agency's  requirements 
as  stated  in  Factor  2 [2]  and  its  method  for  preparing  cost  estimates. 
The  TEP  rated  Tidewater's  BAFO  with  the  same  score  because  the  Form 
477  still  did  not  contain  sufficient  detail  and  because  Tidewater's 
initial  response  regarding  the  preparation  of  cost  estimates  caused 
the  TEP  to  question  the  validity  and  sincerity  of  Tidewater's  response 
in  its  BAFO. 

Our  review  of  Tidewater's  BAFO  indicates  that  the  information 
contained  in  its  Form  477  and  its  overall  response  to  this  factor 
contained  much  the  same  substantive  detail  as  CitiWest's  proposal. 
Based  on  our  review  of  the  two  proposals,  we  cannot  identify  what 
specific  details  are  missing  from,  or  unclear  in,  Tidewater's  Form  477 
with  attachments .   HUD  does  not  explain  what  details  are  missing  or 
what  is  unclear;  nor  does  it  comment  upon  the  accuracy  of  the  needed 
repairs  on  the  specified  property  as  identified  by  Tidewater. 

On  the  other  hand,  while  CitiWest's  technical  proposal  seemed  to 
address  the  same  categories  of  information  for  repairing  a  specific 
property  as  required  in  a  Form  477,  no  Form  477  was  included  in  its 
proposal,  despite  the  RFP's  admonitions,  and  the  property  which  was 
assessed  by  CitiWest  under  this  factor  was  one  that  CitiWest  was 
responsible  for  in  Camden,  New  Jersey  (under  a  REAM  contract 
apparently  administered  by  the  Philadelphia  HUD  office) ,  rather  than 
the  Virginia  Beach  property  identified  in  the  RFP. [3]   We  do  not 
believe  the  evaluators  could  have  reasonably  rated  CitiWest's  proposal 
with  close  to  the  maximum  score  and  much  more  favorably  than 
Tidewater's  proposal  for  this  factor,  given  CitiWest's  failure  to 
provide  a  Form  477  for  the  specified  property  as  was  requested  in  the 
RFP. [4 ] 

Moreover,  we  find  questionable  HUD's  explanation  concerning  its 
failure  to  credit  Tidewater  under  Factor  2  for  its  apparently 
appropriate  BAFO  response  as  to  who  will  prepare  cost  estimates  for 
this  work,  since  agencies  are  generally  required  to  credit  offerors 
for  explanations  in  response  to  discussion  questions.   Intown 
Properties,  Inc.,  supra.   Under  the  circumstances,  the  record  suggests 
disparate  evaluation  of  the  two  proposals  under  this  factor. 

Another  example  of  unequal  treatment  involves  the  evaluation  of  Factor 
6.   The  TEP  awarded  Tidewater's  proposal  a  final  score  of  7  out  of  15 
points  for  this  factor  and  CitiWest's  proposal  12  points.   The  agency 
downgraded  Tidewater's  proposal  because  it  was  allegedly  vague  as  to 
location  of  the  office  space  and  the  division  of  responsibilities 
among  slaff  members.   Here  again,  our  review  indicates  that 
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Tidewater's  BAFO  and  overall  technical  response  to  this  factor  was 
specific  as  to  prospective  locations,  staff,  and  equipment.   In 
contrast,  CitiWest's  proposal  did  not  identify  any  specific  location 
or  staff  in  its  proposal,  but  stated  only  how  it  would  do  so  if 
awarded  the  contract. 

Finally,  Tidewater  argues,  and  our  review  confirms,  that  CitiWest  did 
not  provide  a  Virginia  Real  Estate  Broker  License  with  its  proposal. 
While  CitiWest  proposed  to  promptly  obtain  the  license  after  award, 
Tidewater  notes  that  its  investigation  reveals  that  CitiWest  has  not 
obtained  the  license  and  HUD  has  not  disputed  this  assertion.   In  any 
event,  the  requirement  that  the  offeror  furnish  a  specific  license  to 
be  eligible  for  award  was  a  definitive  responsibility  criterion  that 
had  to  be  satisfied  as  prerequisite  for  award. [5]   RSI  Realty  Servs . 
Inc.,  B-262238,  Dec.  12,  1995,  95-2  CPD  para.   252  (finding  the  same 
requirement  imposed  in  a  REAM  service  procurement  conducted  by  the  HUD 
Philadelphia  office  to  be  a  definitive  responsibility  criterion  which 
was  waived  for  CitiWest) .   By  finding  CitiWest  responsible  and  making 
award  to  that  firm,  the  agency  effectively  waived  the  requirement  for 
possession  of  Lhe  license  prior  Lo  award. [6]   Where  an  ayency  waives 
such  a  requirement,  it  is  required  to  amend  the  RFP;  an  agency's 
failure  to  amend  represents  unequal  treatment  of  the  offerors.   See 
Topley  Realty  Co.,  Inc.,  65  Comp.  Gen.  510  (1986),  86-1  CPD  para.   398. 

In  sum,  as  illustrated  by  the  foregoing  examples,  we  find  that 
Tidewater's  proposal  was  evaluated  unequally  vis-a-vis  CitiWest's 
proposal.   Thus,  we  cannot  conclude  that  the  award  to  CitiWest  is 
reasonably  supported,  and  we  sustain  the  protest  on  this  basis. 
Intown  Properties,  Inc.,  supra;  Sci-Tec  Gauging,  Inc.;  Sarasota 
Measurements  &  Controls,  Inc.,  supra;  Secure  Servs.  Tech.,  Inc., 
supra . 

We  recommend  that  the  agency  determine  whether  the  requirement  that  an 
offeror  possess  a  Virginia  Real  Estate  Broker  License  in  order  to 
receive  award  exceeded  the  agency's  requirements;  if  it  does,  the 
agency  should  amend  the  RFP,  obtain  and  evaluate  new  proposals,  and 
make  award  in  accordance  with  the  revised  RFP.   If  CitiWest  is  not  the 
successful  offeror,  its  contract  should  be  terminated.   Alternatively, 
if  the  license  remains  a  requirement,  we  recommend  that  the  agency 
reject  CitiWest's  proposal  if  CitiWest  does  not  possess  the  requisite 
license  (subject  to  Small  Business  Administration  review  under 
certificate  of  competency  procedures,  see  FAR  subpart  19.6)  and 
terminate  the  contract,  reevaluate  the  remaining  competitive  range 
proposals  in  accordance  with  the  RFP,  and  make  a  new  award  selection. 
In  any  event,  in  light  of  the  evaluation  discrepancies  noted,  we 
recommend  that  a  new  TEP  be  appointed  to  evaluate  the  new  proposals  to 
assure  equal  evaluation.   See  J.M.  Cashman,  Inc.,  B-233773,  Apr.  14, 
1989,  89-1  CPD  para.   380.   We  further  recommend  that  Tidewater  be 
reimbursed  the  reasonable  costs  of  filing  and  pursuing  its  protest 
under  section  21.8(d) (1)  of  our  Bid  Protest  Regulations,  61  Fed.  Reg. 
39043  (1996)  (to  be  codified  at  4  C.F.R.   sec.   21.8(d)(1)).   The  protester 
should  submi L  its  certified  claim  fox  cosLs  Lo  Lhe  conLxacLiny  ayency 
within  60  days  of  receiving  this  decision  pursuant  to  section 
21.8(f)  (1)  of  our  Regulations,  61  Fed.  Reg.  39043  (to  be  codified  at  4 
C.F.R.   sec.   21.8  (f)  (1)  )  . 

The  protest  is  sustained. 
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Comptroller  General 
of  the  United  States 

1.  Our  discussion  of  the  respective  contents  and  evaluation  of 
CitiWest's  and  Tidewater's  proposals  is  necessarily  general  because  we 
recommend  a  reevaluation  of  the  proposals  and  because  no  protective 
order  was  issued,  inasmuch  as  the  protester  did  not  employ  legal 
counsel . 

2.  Tidewater's  BAFO  noted  the  limited  size  on  the  one-page  Form  477 
for  notations. 

3.  The  agency  incorrectly  states  in  its  report  in  response  to  the 
protest  that  CitiWest's  proposal  contained  a  Form  477  "for  the 
property  specifically  identified  in  the  RFP." 

4.  It  appears  that  the  Camden  property  was  in  need  of  much  more 
extensive  repairs  than  the  specified  Virginia  Beach  property,  which 
permitted  CitiWest  to  provide  a  more  extensive  description  and  perhaps 
receive  more  credit  under  this  factor. 

5.  A  definitive  responsibility  criterion  is  a  specific  and  objective 
standard  established  by  an  agency  to  measure  an  offeror's  ability  to 
perform  the  contract.   Federal  Acquisition  Regulation  (FAR)   sec.   9-104-2 
Such  special  standards  put  firms  on  notice  that  the  class  of 
prospective  contractors  is  limited  to  those  meeting  qualitative  or 
quantitative  criteria  deemed  necessary  for  adequate  performance,  e.g., 
unusual  expertise,  specialized  facilities,  or  particular  licenses. 
Tucson  Mobilphone,  Inc.,  B-258408.3,  June  5,  1995,  95-1  CPD  para.   267. 

6.  The  agency  now  states  that  the  requirement  that  the  license  be 
obtained  prior  to  award  overstated  its  minimum  needs  and  that  the 
requirement  as  stated  was  "sometimes  confusing  to  offerors."   The 
protester  notes  that  this  requirement  may  well  have  caused  nonlocal 
firms  not  to  compete  and  required  other  offerors  to  undergo  the 
expense  of  obtaining  the  proper  license  prior  to  award. 


Decision 

Matter  of:  Lloyd  H.  Kessler,  Inc. 

File:  B-284693 

Date:  May  24,  2000 

Jan  D.  Sokol,  Esq.,  Stewart  Sokol  &  Gray,  for  the  protester. 

James  L.  Weiner,  Esq.,  and  Alton  E.  Woods,  Esq.,  Department  of  the  Interior,  for  the  agency. 

Henry  J.  Gorczycki,  Esq.,  and  James  Spangenberg,  Esq.,  Office  of  the  General  Counsel,  GAO,  participated  in  the 


preparation  of  the  decision. 

DIGEST 

Agency  is  required  to  disclose  in  the  solicitation  a  subfactor  to  evaluate  a  particular  type  of  experience  under  the 
experience  factor  where  the  subfactor  constitutes  40  percent  of  the  technical  evaluation 
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Lloyd  H.  Kessler,  Inc.  protests  the  award  of  a  contract  to  John  L.  Jersey  &  Son.  Inc  under  request  for  proposals 
(RFP)  No.  FWSl-00-P004(KH),  issued  by  the  Department  of  the  Interior,  United  States  Fish  and  Wildlife  Service 
(USF WS),  for  river  dike  renovation/rehabilitation  at  the  Ridgefield  National  Wildlife  Refuge,  Washington.  Kessler 
protests  that  the  agency  did  not  adhere  to  the  stated  RFP  evaluation  factors  and  improperly  used  unstated  significant 
subfactors  in  the  evaluation,  and  improperly  evaluated  Kessler's  past  performance. 
We  sustain  the  protest. 

The  RFP,  issued  October  29,  1999,  contemplated  the  award  of  a  fixed-price  construction  contract.  Contract 
performance  cannot  begin  "until  the  eagle  leaves  its  nest"  in  August  2000.  RFP  amend.  0002,  at  1,3.  The  RFP  stated 
a  best  value  evaluation  plan  under  which  "price  and  capability  score  are  of  equal  importance."  RFP  at  53  The 
capability  score  was  said  to  be  based  on  100  possible  points  divided  between  (1)  experience  of  firm  (30  points)  and 
(2)  past  performance  of  firm  (70  points).  RFP  at  45. 

The  RFP's  instructions  for  the  preparation  of  responses  to  the  past  performance  and  experience  factors  stated  that 
"OFFERORS  SHOULD  PROVIDE  ONLY  THE  INFORMATION"  requested  by  the  RFP,  that  is,  specified 
information  about  current  contracts  or  contracts  completed  in  the  previous  2  years,  including  a  brief  description  of 
the  work  performed  and  a  point  of  contact.  RFP  at  46.  The  RFP  stated  that  the  agency  would  request  performance 
information  from  the  contract  references  to  evaluate  quality  of  performance,  and  may  contact,  and  evaluate 
information  received  from,  references  other  than  those  identified  by  offerors.  RFP  at  47.  The  RFP  specified  five 
criteria  that  would  be  considered  in  evaluating  past  performance.  RFP  at  45-46.  The  RFP  stated  no  criteria  that 
would  be  considered  in  evaluating  experience,  but  noted  that  this  evaluation  would  be  based  on  existing  and  prior 
contracts.  RFP  at  45-47. 

The  agency  received  17  proposals  by  the  December  14  closing  date.  Kessler's  proposal  offered  the  lowest  price  of 
$425,880  while  Jersey's  proposed  price  was  $487,221.  Under  the  agency's  evaluation,  Kessler's  proposal  received  a 
score  of  70  points  while  Jersey's  proposal  was  one  of  the  highest-rated  proposals  with  a  score  of  90  points.  Agency 
Report  at  1 ;  Tab  8,  Evaluation  Results. 

The  agency's  actual  evaluation  differed  from  that  stated  in  the  RFP  in  that  the  relative  weights  for  experience  and 
past  performance  were  reversed  and  experience  was  assigned  70  points  and  past  performance  was  assigned  30 
points.  Under  this  evaluation,  the  experience  scores  for  Kessler  and  Jersey  were  55  and  65  points,  respectively,  and 
the  past  performance  scores  were  15  and  25  points,  respectively,  [j  Agency  Report  at  1,  Tab  8,  Evaluation  Results. 
The  scores  for  experience  were  based  on  the  following  three  criteria  and  relative  weights  not  stated  in  the  RFP:  (1) 
bioengineered  slope  protection,  40  of  the  possible  70  points;  (2)  wetland  excavation,  15  points;  and  (3)  rip-rap 
construction,  15  points.  Agency  Report,  Tab  7,  Evaluation  and  Award  Determination  Summary,  at  1.  Kessler's 
proposal  received  25  of  the  possible  40  points  for  the  bioengineered  slope  protection  criterion,  and  received  all  of 
the  combined  30  points  available  under  the  wetlands  and  np-rap  criteria.  Supplemental  Contracting  Officer's 
Statement  (Apr.  10,  2000)  at  5.  The  agency  evaluation  stated  that  Kessler  "apparently  has  average  experience  in 
bioengineering  (2.5  projects  as  the  engineers  can  find  in  the  experience  information  given)  slope  protection  but  has 
good  wetland  excavation  and  rip-rap  placement  experience."  Agency  Report,  Tab  7,  Evaluation  and  Award 
Determination,  at  2. 

Under  past  performance,  Kessler's  proposal  provided  the  requested  information  on  16  contracts  completed  in  the 
past  2  years.  Agency  Report,  Tab  3,  Kessler's  Proposal,  at  16-18.  The  agency  based  its  past  performance  evaluation 
of  Kessler  on  one  of  these  contracts— a  previous  contract  for  dike  repair  with  this  agency —and  two  other  contracts 
not  identified  in  Kessler's  proposal.  Agency  Report,  Tab  5,  Performance  Evaluations.  The  agency  evaluation 
identified  adverse  comments  concerning  Kessler's  performance  that  resulted  in  a  below-average  score;  however,  the 
score  was  increased  to  an  average  score  of  15  out  of  30  points  because  the  agency  did  not  give  Kessler  the 
opportunity  to  respond  to  the  reports  of  adverse  past  performance.  Agency  Report,  Tab  7,  Evaluation  and  Award 
Determination,  at  3. 

The  agency's  cost/technical  tradeoff  analysis  concluded  that  the  concerns  arising  from  reports  of  adverse  past 
performance  for  Kessler  were  not  worth  the  cost  savings  of  Kessler's  lower  price.  Id,  The  agency  determined  that 
Jersey's  proposal,  as  the  lowest  priced  of  the  most  highly  rated  proposals,  represented  the  best  value  Id  at  4,  6. 
On  February  4,  2000,  the  agency  awarded  the  contract  to  Jersey.  This  protest  followed.  The  agency  suspended 
performance  of  Jersey's  contract  pending  resolution  of  the  protest. 

Kessler  protests  the  agency's  failure  to  adhere  to  the  RFP's  stated  evaluation  weights  and  the  use  of  unstated 
experience  subfactors  in  the  evaluation. 

It  is  fundamental  that  offerors  must  be  advised  of  the  bases  upon  which  their  proposals  will  be  evaluated.  H.J.  Group 
Ventures,  Inc..  B-246139,  Feb.  19,  1992,  92-1  CPD  ^  203  at  4.  In  particular,  contracting  agencies  are  required  by 
statute  and  regulation  to  clearly  set  forth  in  the  solicitation  all  evaluation  factors  and  significant  subfactors  that  will 
affect  contract  award  and  their  relative  importance.  41  U.S.C.  §  253a(b)(l)  (1994);  Federal  Acquisition  Regulation  § 
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15.304(d).  An  agency  may  not  give  importance  to  specific  factors,  subfactors,  or  criteria  beyond  that  which  would 
reasonably  be  expected  by  offerors.  Kumasi  Ltd./Kukawa  Ltd.  et  al..  B-247975.7  et  al..  May  3,  1993,  93-1  CPD  ^ 
352  at  6;  Republic  Realty  Servs..  Inc..  B-242629,  May  7,  1991,  91-1  CPD  H  446  at  5.  Contracting  officials  do  not 
have  the  discretion  to  announce  in  the  solicitation  that  they  will  use  one  evaluation  plan  and  then  follow  another 
without  informing  offerors  of  the  changed  plan  and  providing  them  an  opportunity  to  submit  proposals  on  that  basis. 
Kumasi  Ltd./Kukawa  Ltd.. 

et  al.,  supra  at  7.  A  solicitation  that  does  not  set  forth  a  common  basis  for  evaluating  offers,  which  ensures  that  all 
firms  are  on  notice  of  the  factors  for  award  and  can  compete  on  an  equal  basis,  is  materially  deficient.  The  Faxon 
Co..  B-227835.3,  B-227835.5,  Nov.  2,  1987,  87-2  CPD  f  425  at  4. 

Here,  contrary  to  the  terms  of  the  RFP,  the  agency  considered  experience  to  be  the  most  important  technical  factor 
comprising  over  two-thirds  of  the  capability  score.  The  agency  also  states  that  the  single  subfactor,  bioengineered 
slope  protection,  is  the  key  consideration  in  the  overall  evaluation  of  experience,  "as  it  will  make  or  break  this 
project,"  Agency  Report,  Tab  7,  Evaluation  and  Award  Determination  Summary,  at  1  (italics  in  original).  Consistent 
with  this  statement,  bioengineered  slope  protection,  though  the  term  does  not  appear  anywhere  in  the  RFP,  is  the 
single-most  important  technical  criterion,  carrying  more  weight  in  the  unstated  evaluation  plan— 40  out  of  100 
overall  capability  points— than  past  performance,  which  was  stated  to  be  the  most  important  evaluation  factor,  or  the 
weight  assigned  in  the  solicitation  to  the  entire  experience  factor.  This  is  a  significant  evaluation  subfactor  that,  by 
statute  and  regulation,  must  be  clearly  stated  in  the  RFP  along  with  its  relative  importance,  even  assuming  it  was 
considered  reasonably  related  to  the  general  experience  factor  stated  in  the  RFP.  See  Kumasi  Ltd./Kukawa  Ltd..  et 
al,  supra,  at  6  (subfactor  four  times  more  important  than  reasonably  apparent  from  the  RFP  must  be  disclosed  in 
RFP);  Devres.  Inc..  B-224017,  Dec.  8,  1986,  86-2  CPD  ^  652  at  3  (subfactor  worth  more  than  any  other  technical 
factor  is  "significant"); 

cf  Bulova  Techs..  LLC.  B-281384,  B-281384.2,  Feb.  3,  1999,  99-1  CPD  K  99,  at  7-8  (agencies  can  properly  take 
into  consideration  specific,  albeit  not  expressly  identified,  experience  in  making  qualitative  distinctions  between 
competing  proposals,  so  long  as  the  specific  experience  is  logically  encompassed  by  or  related  to  the  RFP's 
requirements  and  stated  basis  for  evaluation). 

The  RFP  did  not  otherwise  indicate  that  "bioengineered  slope  protection"  had  the  overwhelming  significance  given 
in  the  evaluation.  As  noted,  this  term  is  not  mentioned  anywhere  in  the  RFP.  The  term  appears  in  the  Evaluation  and 
Award  Determination,  which  identifies  this  requirement  as  contract  line  item  number  (CLIN)  1 1  in  the  RFP,  which 
is  "Furnishing  and  Placing  Fabric  Slope  Protection."  £J]  Agency  Report,  Tab  7,  Evaluation  and  Award  Evaluation,  at 
1 ;  see  RFP  Bid  Schedule  at  1 .  Neither  the  estimated  quantity  for  this  CLIN  and  its  associated  price  (as  judged  from 
the  protester's  and  awardee's  proposals),  nor  the  description  of  this  requirement  in  the  statement  of  work  indicates  a 
level  of  significance  approaching  the  weight  accorded  to  bioengineered  slope  protection  in  the  agency's  evaluation. 
RFP  at  C- 108- 16;  Agency  Report,  Tabs  3  and  4,  Proposals,  Bid  Schedule  at  1. 

The  agency  concedes  that  the  actual  relative  weights  of  the  evaluation  factors  were  not  consistent  with  that  stated  in 
the  RFP,  but  asserts  that  Kessler  was  not  prejudiced  by  this  evaluation  because  Kessler's  relative  ranking  would  not 
improve  if  the  weights  announced  in  the  RFP  had  been  used.  The  agency's  arguments  do  not  consider  the  fact  the 
proposals  were  prepared  without  offerors  knowing  either  the  overwhelming  significance  of  experience  generally,  or 
bioengineered  slope  protection  specifically,  in  the  evaluation,  such  that  it  cannot  be  said  that  offerors  had  a 
reasonable  opportunity  to  compete  on  this  basis. 

In  this  regard,  Kessler  prepared  its  proposal  under  the  stated  RFP  evaluation  plan  whereby  experience  was  the  least 
important  technical  factor.  As  instructed  by  the  RFP,  Kessler  provided  only  a  brief  description  of  the  work 
performed  under  its  prior  contracts,  and  did  not  specifically  identify  any  work  involving  bioengineered  slope 
protection.  Agency  Report,  Tab  3,  Kessler's  Proposal,  at  16-18.  Even  so,  the  agency  was  able  to  determine  that 
Kessler  "apparently  has  average  experience"  under  this  subfactor.  Agency  Report,  Tab  7,  Evaluation  and  Award 
Determination,  at  2.  Kessler  states  that,  had  it  known  when  preparing  its  proposal  that  experience  would  be  worth  70 
percent  of  the  capability  score,  and  that  bioengineered  slope  protection  was  of  prime  importance  in  the  evaluation,  it 
would  have  placed  greater  emphasis  on  experience  in  the  contents  of  its  proposal,  and  specifically  would  have 
emphasized  its  experience  providing  bioengineered  slope  protection.  Protester's  Comments  at  9-10.  Although  the 
agency  evaluation  determined  that  Kessler  performed  bioengineered  slope  protection  on  "2.5  projects  as  the 
engineers  can  find  in  the  experience  information  given",  [J  Agency  Report,  Tab  7,  Evaluation  and  Award 
Determination  Summary,  at  2,  Kessler  states  that  all  of  its  projects  involved  this  type  of  work  and  it  would  have 
provided  this  and  other  experience  information,  had  the  agency  stated  that  experience  and  this  subfactor  were  as 
significant  as  they  were  evaluated  Protester's  Supplemental  Comments  at  1 1. 

Given  that  Kessler  has  relevant  experience,  it  is  reasonable  to  conclude  that  Kessler's  score  could  increase 
considerably  if  the  firm  is  permitted  to  submit  a  proposal  with  the  knowledge  of  the  actual  importance  of  experience 
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generally,  and  of  bioengineered  slope  protection  specifically.  £]  Because  the  agency  used  an  evaluation  plan  not 

apparent  from  the  RFP,  the  offerors,  including  Kessler,  did  not  have  the  opportunity  to  compete  on  a  common  basis, 

so  there  is  at  least  a  reasonable  possibility  that  Kessler' s  lowest-priced  proposal  will  be  selected  for  award  if  the 

actual  evaluation  factors  and  subfactors  are  disclosed.  [J  We  therefore  find  that  Kessler  was  prejudiced 

We  recommend  that  the  agency  amend  the  solicitation  to  state  the  agency's  proposal  requirements  and  evaluation 

plan,  request  and  evaluate  revised  proposals,  and  make  a  new  source  selection  decision,  {fj  If  a  proposal  other  than 

Jersey's  is  selected  for  award,  the  agency  should  terminate  the  contract  previously  awarded  to  that  firm.  We  also 

recommend  that  the  protester  be  reimbursed  the  reasonable  cost  of  filing  and  pursuing  its  protest,  including 

attorneys'  fees.  4  C.F.R.  §  21.8(d)(1)  (2000).  The 

protester  should  submit  its  claim  for  costs,  detailing  and  certifying  the  time  expended  and  cost  incurred,  with  the 

contracting  agency  within  60  days  after  receipt  of  this  decision.  4  C.F.R.  §  21.8(f)(1). 

The  protest  is  sustained 

Comptroller  General 

of  the  United  States 

Notes 

1.  The  agency  also  assigned  risk  ratings  based  on  the  offerors'  present  and  past  performance.  Kessier's  rating  was 
high  risk  and  Jersey's  rating  was  low  risk. 

Agency  Report,  Tab  7,  Evaluation  and  Award  Determination,  at  2-3. 

2.  In  its  Supplemental  Report,  the  agency  references  other  RFP  work  items  not  included  in  CLIN  1 1  that  constitute 
"bioengineering."  Supplemental  Report  at  2.  However,  the  award  document  states  that  CLIN  1 1  was  the  "more 
weighted  item"  in  the  evaluation  because  it  encompasses  "bioengineered  slope  protection."  Agency  Report,  Tab  7, 
Evaluation  and  Award  Determination,  at  1. 

3.  As  stated,  Kessier's  proposal  did  not  identify  bioengineered  slope  protection  work.  Neither  the  evaluation  record 
nor  the  agency's  responses  to  this  protest  demonstrates  how  the  agency  made  its  determination  in  this  regard  as  to 
Kessier's  proposal  or  any  other  proposal.  We  also  question  what  .5  of  a  project  means  for  purposes  of  this  evaluation 
and  how  that  could  be  determined.  No  other  offeror  was  noted  to  have  experience  on  fractions  of  a  project. 

4.  It  is  possible  that,  under  such  circumstances,  Kessier's  capability  score  could  increase  from  70  to  85  points  from 
the  evaluation  of  bioengineered  slope  protection  alone,  which  would  place  its  lowest-priced  proposal  extremely 
close  to  the  highest-rated  proposals,  including  Jersey's,  thus  changing  the  underlying  basis  upon  which  the  agency's 
cost/technical  tradeoff  determination  and  source  selection  decision  was  made 

5.  We  note  that,  even  beyond  the  agency's  failure  to  disclose  the  actual  evaluation  plan  in  the  RFP,  the 
documentation  supporting  the  evaluation  is  minimal,  such  that  it  would  be  difficult  to  determine  the  reasonableness 
of  the  evaluation,  even  if  the  RFP  were  not  defective.  The  little  documentation  that  does  exist  indicates  that  the 
agency  unequally  evaluated  offerors  under  the  bioengineered  slope  protection  subfactor,  in  that,  although  the 
experience  under  the  other  subfactors  for  Kessler  and  another  offeror  were  evaluated  as  good,  that  offeror  had 
"minimal"  bioengineered  slope  protection  experience  compared  to  Kessier's  "average"  experience,  but  nevertheless 
received  a  considerably  higher  experience  score  than  Kessler.  Agency  Report,  Tab  7,  Evaluation  and  Award 
Determination  Summary,  at  2,  5. 

6.  Kessler  also  contests  its  past  performance  evaluation.  While  we  do  not  decide  whether  the  past  performance 
evaluation  was  improper,  the  record  reflects  some  problems  that  should  be  appropriately  addressed  when  the  agency 
reopens  discussions.  Specifically,  the  record  reflects  that  the  agency  contacted  only  1  of  the  16  references  listed  in 
Kessier's  proposal  (a  USFWS  contract),  and  while  the  agency  alleges  that  the  other  references  declined  to  comment 
on  Kessier's  performance,  there  is  no  documentation  supporting  this  statement,  and  Kessier  has  submitted  evidence 
that  no  such  contacts  were  made.  The  other  two  references  considered  by  the  agency  were  for  contracts  apparently 
completed  more  than  3  years  before  the  evaluation,  although  the  RFP  requested  information  only  on  contracts 
performed  in  the  last  2  years.  In  addition,  Kessler  has  vigorously  disputed  the  accuracy  of  the  negative  comments 
regarding  its  past  performance,  and  the  agency  states  that  Kessler  was  not  given  the  opportunity  to  respond  to  these 
negative  comments.  Since  such  matters  should  be  raised,  where,  as  here,  discussions  are  to  be  conducted,  the  agency 
can  now  afford  Kessler  an  opportunity  to  respond  to  any  negative  comments  regarding  its  past  performance. 
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